What do we understand by Build Verification test?
BVT is - when we get a build to test, we will be verifying whether it is suitable for further testing or not. Using sanity testing and without any formal test cases we will be going through here and there to find any show stoppers. If we are not able to accept the build we will discard this build, switch over to the earlier build (provided there is one) and continue testing the earlier build.Another defination: Build Verification test is a predefined set of tests run on every new build to verify that build is stable before it is released to test team for further testing. These test cases are main functionality test cases that ensure application is stable and can be tested thoroughly. In most of the projects BVT process is automated. If BVT fails that build is assigned
back to developer for fix.Build Verification test is also called smoke testing or build acceptance testing (BAT) BVT is a type of regression testing, done on each and every new build. It is a subset of tests that verify main functionalities of the application. The BVTs are typically run on daily builds and if the BVT fails the build is rejected by the tester and a new build is released after the fixes are implemented.
The advantage of BVT is it saves the efforts of a test team to setup and test a build when major functionality is broken. Design BVTs carefully enough to cover basic functionality. Typically BVT is a 30 minutes to one hour test. BVT primarily checks for the project integrity and checks whether all the modules are integrated properly or not. Module integration testing is very important when different teams develop project modules. I heard many cases of application failure due to improper module integration. Even in worst cases complete project gets scraped due to failure in module integration.
Which test cases are included in BVT?
This is very important decision to take before automating the BVT task. Keep in mind that success of BVT depends on which test cases you include in BVT.
Below are some guidelines to include test cases in your BVT automation suite:
Include only critical test cases in BVT. For example in use case base projects the main flows. All test cases included in BVT should be passed in past.
Make sure all included critical functionality test cases are sufficient for application test coverage. Also do not includes modules in BVT, which are not yet stable. For some under-development features you cant predict expected behavior as these modules are unstable and you might know some known failures before testing for these incomplete modules. There is no point using such modules or test cases in BVT. One you include identify the BVT test cases just make sure it is reviewed by the project management team and approved. Example: Test cases to be included in BVT for Notepad application (Some sample tests only):
1) Test case for creating Notepad file.
2) Test cases for writing something into Notepad editor
3) Test case for copy, cut, paste functionality of Notepad editor
4) Test case for opening, saving, deleting Notepad file.
BVT automation suits needs to be maintained and modified time-to-time. E.g. include test cases in BVT when there are new stable project modules available.
What happens when BVT suite run:
Say Build verification automation test suite executed after any new daily build.
1) The result of BVT execution is saved in the log file.
2) The BVT owner (person executing and maintaining the BVT suite) inspects the result of BVT from the log.
3) If BVT fails then BVT owner diagnose the cause of failure.
4) If the failure cause is defect in build, all the relevant information with failure logs is sent to respective developers.
5) Developer on his initial diagnostic replies to team about the failure cause. Whether this is really a bug? And if its a bug then what will be his bug-fixing scenario.
6) On bug fix once again BVT test suite is executed and if build passes BVT, the build is passed to test team for further detail functionality, performance and other testes.
This process gets repeated for every new daily build.
Why BVT or build fails?
In daily build Dev team giving fixes of bugs. There is a high chance that when they fix something in the application, some other part get broken and new bug introduced. This ma lead to a situation where one of the main application part is not working which was working earlier.BVT breaks sometimes. But there can be other errors as well such as test case coding error, automation suite error, infrastructure error, hardware failures etc.
You need to troubleshoot the cause for the BVT break and need to take proper action after diagnosis.
Sample BVT acceptatnce criteria:
The BVT must have successfully concluded all planned testing including Manual and automated. The following parameters will determine the failure of BVT:
1) 85 to 90% priority one/two defects mentioned fixed in the release note still not resolved
2) The severity of defects prevents any further testing
3) 85 to 90% of the regression should pass.
4) Not more than one severity defects are introduced in the build.
Conclusion:
BVT is nothing but a set of regression test cases that are executed each time for new build. This is also called as smoke test. Build is not assigned to test team unless and until the BVT passes. BVT can be run by developer or tester and BVT result is communicated throughout the team and immediate action is taken to fix the bug if BVT fails. BVT process is typically automated by writing scripts for test cases. Only critical test cases are included in BVT. These test cases should ensure application test coverage. BVT is very effective for daily as well as long term builds. This saves significant time, cost, resources and after all no frustration of test team for incomplete build.
Subscribe to this comment's feed
| < Prev | Next > |
|---|
