Software testing tools can improve test efficiency, reduce test execution time and improve the software quality. However, for new people in test automation, there are many one-sided misunderstandings about the role of software testing tools in a project. Let us discuss them one by one:
Misunderstanding one: the use of tools can find all or most of the software defects.
In fact, more than 80% defects are found during manual testing and only 20% defect can be found by using tools. Test tools are used mainly during regression testing and performance testing. In regression testing phase, the application is supposed to be very stable and using an automation tool you make sure the main functions are working fine. Since this time, the code change is thin and no major changes happen so it is unlikely that the tool will find lots of defects. Performance testing using tools can find some major defects but again compared to the functional test defects this number is nominal.
Misunderstanding two: one software testing tool can meet all your automation requirements.
Test tools are developed to address specific problems, so inevitably has its limitations. Software testing tools are also software, so they will have also some bugs. So you need different tools to solve different problems. A performance test tool cannot do functional automation or vice versa. A tool developed specifically for web automation cannot be used to automate a desktop application. So it is important to research on test automation tools based on your application type before you make any purchase.
Misunderstanding number three: the use of software testing tools will immediately reduce the test execution time and hence reducing the overall cost.
Adding a new element in the testing process will increase the complexity of the testing process. Generally, in the early stage of test automation, the cost of testing will be increased by 25% to 50%. However, eventually when the automation suits are ready and reusable, you can reduce the execution cost to some great extent. In one project, it used to take 80 man days to run a full test cycle but using proper automation it was reduced to three days because the tool can run the jobs in the background in 24×7 manner.
Misunderstanding four: test tools can be started quickly, and does not require specialized training and learning.
Many vendors have tried to exaggerate the tool easiness to sell their products, and pointed out that their tool can be used in a simple record and play manner. However, effective automation is not so simple, during recording a tool generated test scripts must be manually modified, which requires good knowledge of test scripting, so that the script robust, reusable and maintainable.
Common scripting languages are java script, vb script, or Perl etc.. Therefore, the use of any new testing tools requires professional training and learning.
Misunderstanding Five: Using a testing tool, we can achieve 100% test coverage.
The tool can increase the depth and breadth of test coverage, but you cannot rely on software testing tools to get 100% test coverage. As a matter of fact, many good major defects are found by Ad-hoc testing which a tool cannot do.