Software Testing Social Network

A testing methodology is a tool or method used to test an application. Some methodologies include monkey testing, automated UI testing, regression testing, and so forth. Some might argue that testing techniques such as pairwise-combinatorial interdependence modeling or model-based testing are also methodologies. A testing strategy, on the other hand, is a holistic view to how you will test a product -- it's the approach you will take, the tools (and methodologies) you will use to deliver the highest possible quality at the end of a project.

In soccer/football, teams use methodologies and strategies. Some strategies include double-teaming the opponent's most aggressive player, choosing a more defense-oriented lineup, or keeping the game pace slow with a team known for rapid strikes. Methodologies frequently define how these strategies are implemented. For instance, having two mid-fielders (halfbacks), with one being more defense-minded, would allow a team to double-team a world champion center mid-fielder. Continually pushing the ball to the outside and leveraging short passes in the classic triangle is a methodology for implementing a slower approach to the game.

In software quality, the test strategy consists of a myriad of methodologies, activities, and staffing solutions. The strategy overall sets the acceptable bar and calls out how the test team will achieve that bar. It is the sum of all the inputs, in an organized plan. Testing methodologies are the different approaches you will take to testing. Currently, I'm testing a new Web site (available in late summer). The site has two major components: a UI-oriented content component for external visitors and a Web services component used by a third-party provider to interact with an internal database.

My test strategy calls for implementing two entirely different methodologies for testing this project. The content-heavy site includes a fair amount of manual UI-based testing, ensuring the quality of the content as well as the flow and lay-out of content templates. It includes (and for future regression it relies heavily on) a suite of Selenium-based tests which validate divs on the page, page headers, and successful page rendering. A base set of page tests has already been written, which checks for a series of positive and negative tests (for instance, it looks for the present of headers and footers; it also looks for 404 responses, for script errors, or for server errors on each page regardless of the URL or page content). We have also written a series of manual test cases to test the business logic on forms pages. Finally, we've inherited a set of test cases for testing the search component, because it is similar to the search component on another site we own. Eventually each page will have a set of manual and automated test cases, developed in conjunction with the page development.

The Web services component, however, uses an entirely different testing methodology. We are relying on eviware's SoapUI Pro Web services testing project (http://www.eviware.com), and we are implementing a data-driven testing methodology. Currently our first Web methods are trickling in for the service and we are engaged in writing the plumbing for data-driven testing of each Web method. Once the plumbing is in place, we can add new tests (positive or negative) simply by filling in a row of test data including input, expected response, and result. We're nearly 100% automated in our approach to testing the Web service.