To build a functional testing framework we need to define the following software testing artifacts at the very first stage of the project.
1. Test Plan: This needs to be written in a very early stage of the project (typically at the time of project design phase). The test manager needs to sit with all the project stakeholders and client to get it done, reviews and signed off.
Typically it should contain the following important things:
i. The total number of the resources (tester, test lead and test manager) needed to complete the project. So a clear stuffing plan must be determined.
ii. The test schedule is another important part of the test plan. Ideally it should not be a very tight. Put some buffer time in place. In many cases you will see delay in the original testing due to development delay or decision pending from client so if you do not put some buffer time those unseen risks and issues can not be handled and you end up doing lots of work to meet the schedule.
iii. Test environment configuration: Clearly define the hardware and software needs, their license procurement procedure etc. If you are working in offshore and onshore model then mention the access restrictions to the onshore systems from offshore if there is any, the performance issue, do you need extra bandwidth to access the onshore servers etc.
iv. Write down the items to be tested, items which are beyond the scope of the testing.
v. Clearly define a knowledge transfer plan for the system you are going to test. It may need you to send some tester to onsite to learn the system and understand the requirements.
vi. Identify the test exit criteria or when you can stop the testing. It varies from client to client. But in the SOW/contract document it should be clearly mentioned that when you can say the testing is done.
2. Test cases: The test case documents tell you how to do the testing for a functionality. Each requirement should have ideally 4 test cases associated with it viz two boundary values, one negative test and one happy path testing. It should have the test objective/business scenario, test steps which tells how to perform the testing, test data, expected result and actual result mentioned.
3. Defect Report guide: this is the most commonly used document for any functional testing project. Any deviation from the original requirement should be logged as a defect. A defect incident report should contain the following things in minimum:
* Defect ID: Defect ID number which is unique. No duplication of defect ID’s possible
* Reporter’s/Originator Name on the bug. If there are questions we need to know who originated this report.
* Assigned to (This bug is assigned); who’s going to be responsible for the bug and do the work on the bug?
* Defect Status State of the defect like submitted, assigned, planned, completed, re-tested, closed etc.,
* Defect Type Defect Type may be logical, Data, Documentation, Bad fix, Third party, Performance etc.,
* Date of origin Date of origin of the defect like when it was found.
* Build or Version number This gives the details of the code being worked on.
* Phase found The defect found in (like Requirement, design, coding, testing) phase.
* Origin of defect The defect originated in (like Requirement, design, coding, testing) phase.
* Severity and Priority of the defect – Severity (High, medium, low) and Priority (High, medium, low) of the bug
* Brief Description of defect (what the problem is) For example, “Fatal error when printing landscape.” is a good description; short and to the point
* Platform on which the bug found on
* Windows, Linux, etc. Is the bug specific to one platform or does it occur on all platforms?
* Feature or Specification or part of the code.
* This facilitates assigning the bug to a developer assigned to that part of the product.
* Test case No. Test case reference number given here like Test_case v1.0 (where the defect is found)
* Details including how to reproduce the bug and any other relevant data or clues about the bug.
– Start with how the computer and software is setup.
– List each and every step (don’t leave any out) to produce the bug.
– Attach Snapshot of the error.
– Sometimes a minor detail can make all the difference in duplicating or not duplicating a bug. For example, using the keyboard versus using the mouse may product very different results when duplicating a bug.
– Attach the screenshot of the error if there is any
4. Test Summary Report: This contains the report and summary of the project test results, citing the relevant test data and statistics, and record the remaining issues. It typically contains total number of defects logged in the testing phase, the severity level of those defect, defects type and how many of them are still open or deferred to the next release.Also test exit report/summary report is an official sign off from the test team.