Today's enterprises need to control their business-critical applications in all functional tests to ensure that all business processes work as expected. By implementing automated functional testing, companies can dramatically improve the test speed and accuracy and get higher return on investment and significantly reduce risk. This paper briefly describes the advantages of automated functional tests and challenges, to help companies considering the implementation of the best test automation approach.
1. Introduction to Automated Functional Testing:
There is no doubt that the strict application of functional testing is the key to successful development. Developers, test team and management’s challenge is how to speed up the testing process and improve the testing accuracy and completeness. But also they can not increase the tight budget of the project.
By a key tool called functional test automation one can meet the challenging schedule of the application release, test more comprehensively and increase the reliability of the correctness of business process functions and add high value to customer satisfaction. However, the functional test automation will have some new concerns:
The testing process automation cost? Their return on investment (ROI). What is this?
What applications / processes suitable for automated testing, and which inappropriate?
The need for automation tool training. What would be its impact in the current development plan? What Automated testing methodology should be followed? Automated testing should be involved in what situation? When doing the automated testing of the application under test, which features are most important?
Before the start of any automation project all of above and other issues should be fully investigated and understood.
2. Functional testing
Functional testing is to ensure that applications run according to the user's expectations. Functional testing is an effective way to capture the user's needs. It is done processes from the point of view of end user and business.
3. Why automate functional testing?
Now, IT departments under increasing pressure. Management hopes that IT departments can deliver new functionality through software, to seize new business opportunities and provide competitive advantage. This implies the need to do more business applications development projects, and time will be very tight and budget or resources are limited.
At the same time Web Services, online transaction processing and ERP applications are not only critical, but, they are directly related to the company's production capacity. Now companies are very dependent on very complex computer infrastructure. Figure, a typical business may depend on multiple applications running on different systems, using several different front-end client, involving a large number of business processes and interact with a very wide range of data sets.
Possible combinations are highly complex, requiring hundreds of thousands of test scenarios.
Components Amount Software/Hardware
Platform 1 Intel
OS 5 Windows XP, ME, 2000, NT4, and 98
Front-end client 4 Internet Explorer 6, Netscape 7.1 Java, Visual C + +
Business Process 5 Login, Search, Order Entry, Order Confirmation, Order Fulfillment
Dataset 15 usernames, passwords, search strings, order numbers, ship dates, and so a combination of
The number of test requirements 1x 5 x 4 x 15 = 1,500 possible test scenarios!!
When the software fails, the cost is very large, including the decline in sales, employees, low efficiency, customer dissatisfaction and the development and QA staff morale. In the software development cycle the cost of a defect becomes higher as they move to the end stage. A design shortcoming related defect found in post production can cost 100 times than that defect found in SIT. Automation is to improve the software testing process, the speed, accuracy and flexibility of the testing of critical business applications which allows companies to early detect the software bugs.
4. Manual functional testing challenges
In Manual functional testing process itself, there are many challenges:
Because of the limited IT resources and the tight delivery time manual testing is too time-consuming to meet business objectives. In manual testing, testing and development have had to plan every step of a lengthy testing process. So during the testing reproduction of defect is a time consuming task. Statistics shows from an independent industry analyst firm that for 90% of IT project delivery delays, manual testing is one factor.
Again in manual testing the test coverage is incomplete. Because of the time and budget constrains if is not possible to test all the Platform, operating system, customer premises equipment, business processes and data sets combination during testing.
Because of the time constrains we take only a subset of functional test cases for regression testing so it never covers all the business functionalities.
In manual testing the risk of human error is quite high. People may give wrong data input or get tired and execute the test case in wrong way.
5. what are the benefits of automated testing?
Automated testing has many advantages, including:
Fast Testing: Computer can test much faster than human. So in a limited period of time more applications can be tested. Also the computer 24 hours a day's work also includes evenings, weekends and holidays; they will not feel bored or tired; So productivity will be very high.
To improve test coverage: Through integrated data-driven form features, automated testing tools allows development and testing team to perform calculations, operational data sets, as well as quickly create a variety of repeated tests, making expansion of test coverage. This will improve test accuracy and helps in early detection of more errors
Improve the reusability of the test cases: In the script-based test, developers and testers can reuse these automation scripts, the script can be added to the test suite in order to adapt to changes in the application. There is no need for each application to re-create the same functionality script.
6. When to use automated testing?
In general, the automated testing of the work focuses on key business processes, complex applications, as well as the composition of use cases from these areas (as opposed to low-level tasks, such as system-level verification) is of great significance.
If an enterprise has many testers working on software testing but the product quality and functionality is still the same then surely this enterprise can benefit from automated testing. Decision on whether to implement automated testing should fully take into account the return on investment. If an application requires multiple patch / modification; need a lot of hardware or software configuration under test; and support of many concurrent users, etc., then automated testing would be worthwhile. In addition, if test cases have repetitive steps, such as data loading and system configuration, or the application needs to meet specific service level agreement (SLA), then the automated testing of course will save some cost.
7. How to determine return on investment (ROIS) of automated testing?
Any return on investment can be derived from a simple calculation:
Return on investment = net present value of investment / total initial cost
When using the automated testing process, the costs are tangible, but the net present value still includes many intangible factors. The best way is to try to accurately calculate the direct costs as well as the indirect cost.
ROI calculation in the direct costs needs to include:
Purchase cost: purchase cost of automated testing software products.
Hardware costs: Some automation test software tools are resource intensive and may need extra RAM, high end processor etc. Particularly for performance testing we need more high end machines so this is another direct cost associated with any automation testing.
Labor costs: Automation needs staff training on the preparation of test scripts or manual test case writing. So we need to include cost of new resource recruitment, people wages and training cost etc. If trained stuffs are not present then companies may choose to hire professional services firms to create the first automated testing.
When we measure the potential benefits of automation, we must consider its hidden benefit such as testing of high staff morale and job satisfaction, improved customer satisfaction and loyalty and the continuous improvement visibility.
8. How to assess the Automated Functional Testing Tools?
Many vendors provide automated testing products. Each tool has its own advantages and disadvantages, unique features, and the market environment. So choosing the right automation tool mainly depends on the organization needs. However, when evaluating automated functional testing tool we should check a number of key areas:
Automated testing of the "Scriptless" notation: Product should provide a clickable interface from where one can run the automation script rather than showing the script line by line. Testers should be able to visualize each step of the business processes, and can edit test cases from there.
Integrated Data Sheet: A key advantage of automated functional testing is that the system can quickly produce large amounts of data. Another important feature is the operation on the data sets, calculations and with minimal cost quickly creates hundreds of duplicate tests and combinations. Enterprises should look for vendor to provide this kind of test data creation and management feature.
Good reporting of test results: If the test results are not easy to understand or explain, even if we are running a large number of test data would not be any good. Test tool should automatically generate and display all aspects of the test run report and interpret the results with easy to read format. Report should provide details, including: failure occurred in which part of testing and used test data for that failure;
9. Points list: automated functional testing of the five key success
Even if tit is proven automation is cost-effective, but transition of manual to automation is still quite difficult. This section outlines five basic principles of the implementation of automated testing process.
1. The completion of an automation test plan document. This includes a comprehensive plan in advance to ensure that the testing requirements are properly implemented. All the testing tool and management related requirements needs to be captured here.
2. How much automation: It is not possible in an organization to automate all of the test cases. Automated testing should focus on the critical test cases only. A good rule of thumb is to automate only 60% of the total test cases, while the remaining 40% will be manual testing.
3. Selection of test tool: Test tool greatly simplifies the preparation process of test data and scripts. This allows for more complete test to optimize the use of test resources and results. Testing tool should be able to automatically capture the target application's business processes, and allows users to create reusable automation scripts which can be saved for later use.
4. To improve test coverage of the data-driven testing. Testers can use test tool to create and store various types of test data and store in the Excel spreadsheet. This allows testers to execute the application flows with a large amount of test data.
5. Increase the validation test. Need to add various "pass or fail" checkpoint to the test. This includes the checkpoint at application front-end, middle tier, or back-end database verification. For example in the database, verify that the database value is stored properly, and has been updated, deleted, or the increase in the number of data records etc.
10. Summary
Manual functional testing is time-consuming or costly. In automated functional testing, companies can focus on improving the automated business processes. This will increase the speed and accuracy of the development and QA team’s testing process. The entire IT departments can be a higher return on investment, but also reduces a great deal of risk.
Subscribe to this comment's feed
| Next > |
|---|
