Software Testing Social Network

LoadRunner is a tool, or more accurately a collection of tools, which facilitates in
performing load testing on a system. LoadRunner is used to create a load or stress on a system, and can be used to measure the responsiveness of the system to that load. The key concept behind LoadRunner is the use of virtual users, or vusers.

What is "Virtual Users"?

To perform a load test, a system needs many users to access the system concurrently, often one hundred or more. Performing such a test with people is a logistical nightmare. When considering that the test cycle is generally iterative, this problem is compounded.
LoadRunner overcomes this problem through the use of vusers. A vuser performs the actions that a real user would perform, except that a vuser is controlled by LoadRunner, and the actions are performed automatically.
Vusers are created to perform specific transactions in the system. Once created a vuser can be replicated as many times as required, thus producing an effective load as large as is required. With the exception of GUI vusers, virtual users don’t use the client to perform their actions; they take the place of the client. This allows in many vusers to be run on one workstation, as the overhead associated with the client is not required.
This does mean, however, that non-GUI vusers do not test the performance associated with the client side of a system, only the network and server associated performance.

Vusers can be of four main types. 

DB Virtual Users
A DB, or database vuser performs operations against a database server. LoadRunner supports all the most popular interfaces to database servers including but not limited to ODBC, SQL, and various Oracle interfaces. It is also possible to create LoadRunner interfaces that allow recording against unsupported databases. Recording a DB vuser is as simple as performing a business process while having launched the LoadRunner Virtual User Generator. This creates a script which is used by LoadRunner to perform the recorded business process.

Once recorded, a vuser script is then parameterised. This process involves replacing the hard coded data values in the script with parameters, which refer to tables external to the script. Values can then be placed in these tables allowing vusers to perform a single business process, but with different data each time.

Web Virtual Users
A web virtual user is one that performs actions against a web based system. A web vuser is essentially identical to a DB vuser, except that where the DB vuser is recorded against a client application that communicates directly with a database of some description, a web vuser is recorded against a web based application.

RTE Virtual Users
Remote Terminal Emulation virtual users are used to emulate users that would be operating on a lightweight, character based terminal.

SAP Virtual Users
SAP R/3 virtual users are created somewhat differently than the others, and require the SAP R/3 LoadRunner add-on. Essentially, a SAP R/3 virtual user is one that performs actions in a SAP R/3 environment.

GUI Virtual Users
Unlike the previous types of vusers, GUI virtual users actually perform actions using the client. This means that fewer GUI vusers can be run per workstation than the other types of vusers due to the overhead of the client, and is in fact limited to one GUI vuser per PC workstation.
GUI vusers are used not so much to generate load on a system, but to test end-to-end performance associated with various business processes. Although LoadRunner can control GUI vusers, it is unable to create them. For this purpose, WinRunner is required.

LoadRunner is a tool, or more accurately a collection of tools, which facilitates in
performing load testing on a system. LoadRunner is used to create a load or stress on a
system, and can be used to measure the responsiveness of the system to that load. The
key concept behind LoadRunner is the use of virtual users, or vusers.

What is "Virtual Users"?

To perform a load test, a system needs many users to access the system concurrently,
often one hundred or more. Performing such a test with people is a logistical
nightmare. When considering that the test cycle is generally iterative, this problem is
compounded.
LoadRunner overcomes this problem through the use of vusers. A vuser performs the
actions that a real user would perform, except that a vuser is controlled by
LoadRunner, and the actions are performed automatically.
Vusers are created to perform specific transactions in the system. Once created a vuser
can be replicated as many times as required, thus producing an effective load as large
as is required. With the exception of GUI vusers, virtual users don’t use the client to
perform their actions; they take the place of the client. This allows in many vusers to
be run on one workstation, as the overhead associated with the client is not required.
This does mean, however, that non-GUI vusers do not test the performance associated
with the client side of a system, only the network and server associated performance.
Vusers can be of four main types.

DB Virtual Users
A DB, or database vuser performs operations against a database server. LoadRunner
supports all the most popular interfaces to database servers including but not limited to
ODBC, SQL, and various Oracle interfaces. It is also possible to create LoadRunner
interfaces that allow recording against unsupported databases.
Recording a DB vuser is as simple as performing a business process while having
launched the LoadRunner Virtual User Generator. This creates a script which is used
by LoadRunner to perform the recorded business process.
Once recorded, a vuser script is then parameterised. This process involves replacing
the hard coded data values in the script with parameters, which refer to tables external
to the script. Values can then be placed in these tables allowing vusers to perform a
single business process, but with different data each time.

Web Virtual Users
A web virtual user is one that performs actions against a web based system. A web
vuser is essentially identical to a DB vuser, except that where the DB vuser is recorded
against a client application that communicates directly with a database of some
description, a web vuser is recorded against a web based application.

RTE Virtual Users
Remote Terminal Emulation virtual users are used to emulate users that would be
operating on a lightweight, character based terminal.
SAP Virtual Users
SAP R/3 virtual users are created somewhat differently than the others, and require the
SAP R/3 LoadRunner add-on. Essentially, a SAP R/3 virtual user is one that performs
actions in a SAP R/3 environment.

GUI Virtual Users
Unlike the previous types of vusers, GUI virtual users actually perform actions using
the client. This means that fewer GUI vusers can be run per workstation than the other
types of vusers due to the overhead of the client, and is in fact limited to one GUI
vuser per PC workstation.
GUI vusers are used not so much to generate load on a system, but to test end-to-end
performance associated with various business processes. Although LoadRunner can
control GUI vusers, it is unable to create them. For this purpose, WinRunner is
required.

Static Testing – An Effective Method Between Coding and Testing Phase

Software testing connotes real-time execution of software beta versions. However, real testing already starts even in coding phase, when developers present their initial code for review. The system is being tested statically once the reviewer began to analyze the code, and ends until a satisfactory code has been successfully compiled. In actual practice, static testing purposes are as follows:

Existential Check of Code Implementation for All User Requirements
This is one of the top priorities of code reviewers or static testers. They make sure that all requirements written in software specifications are implemented in the code. It also extends to check if hidden requirements are met.

Syntax Validation
The main focus of this testing is to have a theoretically working source code, and one way to do this is to ensure that code is syntactically correct. Input arguments for both customized and built-in functions can also be included in validation. Syntax checking is very important because having a compiled source code is the first step in acquiring a working prototype.

Improving Code Readability
Various developers have differing programming style. Having various developers in one source code might result to generally unreadable code, that is, logic flow within the code is difficult to follow and to maintain. Verifying the code statically calls for searching for an alternative solution to be lengthy, hard to follow, and difficult to manage code.

Developing Source Code Efficiency
Aside from well-structured design, efficiency is also achieved through proper implementation of the design. One data type will work better than others and choosing the best type for the entire code is crucial. In addition, poor coding implementation results to longer run-time. This problem can be solved by optimizing the code structure during reviews.

Creating Well-Documented Source Code
Code comments are often neglected in actual practice. Having sufficient comments can help developers to modify the work of each other and to have a more understandable source code. Static testers also aim to utilize modern methodologies, like XML tags, for better code documentation.

Modern static testing techniques offer a more benefit to the software development team. The advent of pair programming, a technique used in extreme programming and Agile process, paved the way for shorter development and review time. Some benefits of modern static techniques are higher quality, lower cost, and better knowledge transfer. It also has its disadvantages like intimidation issues when pairing more experienced developer with a newcomer. Such static technique can also help in skill improvement because studies show that developers increase their productivity as much as two times after participating in several pair programming.

The primary problem encountered in the static analysis is the fact that no method can give exact answers on whether the code will have run-time error or not. To resolve this issue, several formal methods have been developed, and some static testing implementation techniques include:

Model Checking
The system is checked if requirements are met by using a simplified model of its structure. It also includes elimination of situations where two different operations temporarily pause for their actions to wait for the other operations to finish, or commonly known as deadlock.

Abstract Interpretation
Computable mathematical representation of software behavior should be acquired to allow static testers to analyze its theoretical results and expected system action. If done efficiently, static analysis can be “sound”, or has the same result for both abstract and actual system.

Assertions
This is actually a programming keyword used in several code checker utilities. Assertions can validate if the output of the static system is the same with the expected value of the developer.

Advanced implementation can include process champions, trainers, and static analysts. They also use materials like policies, logs, metrics, etc.

Video games are also considered as software that needs sufficient testing before releasing in the market. In fact, many software developers want to become game developers.

Game testing is the process of verifying game environment and behavior. Its occurrence in game development depends on its budget. High-budgeted projects test their products when the draft version is finished. On the other hand, low-budgeted games might only support testing of the probable final version. Other types like public betas and stress tests are also used in actual game industry, but it less systematic and scientific because testing is being done by simply playing the game from the start to finish and reporting random bugs that occur and detected accidentally. Scientific and methodological approaches in testing have the following purposes:

Game Functionality

This includes gaming environment, textures, and elements. Issues like stability and game messages are also included in this purpose. The exciting part about functional testing is verifying the correctness of game elements, because for example, checking if weapons like swords can really incur damage to enemies and of course, the most practical way to do it is to play the actual game.

 Game Compliance

Most games are not stand-alone system. They need to interact with hardware devices and this is especially true for console games. Protocol compatibility and hardware pin assignments should be compatible for both software and hardware side. This is often seen as an area outside the scope of game testers because it now deals with issues beyond the gaming environment.

 Game Localization

It is  the common knowledge that most RPGs and other story-based games came from Japan. Thus, scripts may have inconsistencies when translated from one language to another. Unlike functional testers who need to play the actual game, localization testers often face a spreadsheet-like interface where all game scripts and its translation are listed.

 Game Soaking

Soaking simply means exposing the game for an exaggerated long period of execution. This is to test if the game will exhibit unreleased segments of memory and mathematical rounding discrepancies. Rounding errors can be problematic for games because it can affect its pointing system, health value consumption, and other game parameters that involve numerical computation.

 Games need thorough evaluation to deliver maximum playing experience. For this reason, its testing focuses on some crucial part and breakdown to achieve realistic game behavior. Listed below are some game testing components:

    * Menus, sub menus, and its functions
    * Graphics, texture, and animation
    * Character audio, sound effects, and special effect
    * Game difficulty level
    * World, dungeons, and map settings
    * Player, action, and inanimate attributes
    * Artificial intelligence for the defense and offense mode
    * Game flow, logic, and sequencing
    * Life points, magic points, and player points
    * Game pad vibration, analog stick, and other hardware issues

Game testing tips and considerations are greatly focused in game graphics. For example, testers should verify a realistic behavior when multiple objects overlapped with each other. Collision effects between two objects should also be realistic. This is especially true for games involved in sports. In terms of considerations, testers should be aware of the so-called crack bugs and placeholder. This often appears in the development period. Crack bugs are also called false bugs because it is just the result of misjudgment by the tester, while placeholders are shapes used to cover temporarily some part of the game screen while the real image is not yet finished.

<!-- /* Font Definitions */ @font-face {font-family:Calibri; mso-font-alt:"Century Gothic"; mso-font-charset:0; mso-generic-font-family:swiss; mso-font-pitch:variable; mso-font-signature:-1610611985 1073750139 0 0 159 0;} /* Style Definitions */ p.MsoNormal, li.MsoNormal, div.MsoNormal {mso-style-parent:""; margin-top:0in; margin-right:0in; margin-bottom:10.0pt; margin-left:0in; line-height:115%; mso-pagination:widow-orphan; font-size:11.0pt; font-family:Calibri; mso-fareast-font-family:Calibri; mso-bidi-font-family:"Times New Roman";} @page Section1 {size:8.5in 11.0in; margin:1.0in 1.25in 1.0in 1.25in; mso-header-margin:.5in; mso-footer-margin:.5in; mso-paper-source:0;} div.Section1 {page:Section1;} -->

Video games are also considered as software that needs sufficient testing before releasing in the market. In fact, many software developers want to become game developers.

Game testing is the process of verifying game environment and behavior. Its occurrence in game development depends on its budget. High-budgeted projects test their products when the draft version is finished. On the other hand, low-budgeted games might only support testing of the probable final version. Other types like public betas and stress tests are also used in actual game industry, but it less systematic and scientific because testing is being done by simply playing the game from the start to finish and reporting random bugs that occur and detected accidentally. Scientific and methodological approaches in testing have the following purposes:

Game Functionality

This includes gaming environment, textures, and elements. Issues like stability and game messages are also included in this purpose. The exciting part about functional testing is verifying the correctness of game elements, because for example, checking if weapons like swords can really incur damage to enemies and of course, the most practical way to do it is to play the actual game.

 Game Compliance

Most games are not stand-alone system. They need to interact with hardware devices and this is especially true for console games. Protocol compatibility and hardware pin assignments should be compatible for both software and hardware side. This is often seen as an area outside the scope of game testers because it now deals with issues beyond the gaming environment.

 Game Localization

It is  the common knowledge that most RPGs and other story-based games came from Japan. Thus, scripts may have inconsistencies when translated from one language to another. Unlike functional testers who need to play the actual game, localization testers often face a spreadsheet-like interface where all game scripts and its translation are listed.

 Game Soaking

Soaking simply means exposing the game for an exaggerated long period of execution. This is to test if the game will exhibit unreleased segments of memory and mathematical rounding discrepancies. Rounding errors can be problematic for games because it can affect its pointing system, health value consumption, and other game parameters that involve numerical computation.

 Games need thorough evaluation to deliver maximum playing experience. For this reason, its testing focuses on some crucial part and breakdown to achieve realistic game behavior. Listed below are some game testing components:

• Menus, sub menus, and its functions
• Graphics, texture, and animation
• Character audio, sound effects, and special effect
• Game difficulty level
• World, dungeons, and map settings
• Player, action, and inanimate attributes
• Artificial intelligence for the defense and offense mode
• Game flow, logic, and sequencing
• Life points, magic points, and player points
• Game pad vibration, analog stick, and other hardware issues

Game testing tips and considerations are greatly focused in game graphics. For example, testers should verify a realistic behavior when multiple objects overlapped with each other. Collision effects between two objects should also be realistic. This is especially true for games involved in sports. In terms of considerations, testers should be aware of the so-called crack bugs and placeholder. This often appears in the development period. Crack bugs are also called false bugs because it is just the result of misjudgment by the tester, while placeholders are shapes used to cover temporarily some part of the game screen while the real image is not yet finished.

The only universal statement that will satisfy the diverse software development principles of IT professionals is that the software to be delivered to clients must have the highest accuracy and reliability as possible. Software testing was proven to be a vital part of achieving such results since the early days of IT industry up to present situation when numerous testing methodologies have already been developed. One of the most widely adopted software testing techniques is all-pairs testing.

In this method, input parameters will be combined with one another to create all discrete possibilities that may occur during operation. Contrary to exhaustive testing, or verifying all combinations individually, this method utilizes statistical principles to find software faults and to reduce a number of test cases while maintaining satisfactory test coverage. In actual practice, this often translates to higher productivity and lower cost.

Real-life software testing always faces the dilemma of having thousands of test cases to thoroughly verify internal and external system behavior. However, meeting client needs often restricts project management in having the luxury of time during testing phase to cover all test cases. Thus, different types of all-pairs testing was formed to incorporate both past and modern statistical concepts into test case creation to find as many faults as possible out of minimal cases.

Combinatorial Testing
This testing is the simplest pair testing. In this type, all combinations should be evaluated by tedious approaches like prime-based solutions and brute-force. Such solutions where proven to be tiresome and least reliable because it relies on random probabilities and guesses from previous results. Apart from this, it imposes higher chance of error because of its heavy testing procedures and its similarity with exhaustive testing.

Pairwise Testing
In exhaustive testing, all combinations are being formulated. For example, 243 items should be formulated for a system with five distinct parameters having three possible states. Testers do not have enough resources to cover all 243 items but through the pair wise testing, only minimal test cases will be needed to cover all interactions. Finite field computation, modulo arithmetic, etc. is used to solve the least number of test cases needed. This testing started as early as 1990s and continues to become popular because of its feasibility and high quality despite its low cost.

Orthogonal Array Testing
This type was derived from industrial engineering concepts of manufacturing. It offers robust methodologies in creating efficient test cases. This pair testing is similar with the pair wise but goes further by arranging each combination orthogonal. In the orthogonal arrangement, combinations of values between parameter pairs will only occur once in all sides of the array. This arrangement makes it suitable for more sophisticated software applications because of faster and more reliable bug detection.

Through pair testing is appropriate for most systems, it will not be as efficient when applied in medical equipment, distributed computing, and shared databases. All-pairs testing common mistakes can be summarized into three:

Manual Pair Creation
Using spreadsheets to manually plot pair values is very tedious. It is also difficult to maintain and presents lots of inaccuracies. To avoid such scenarios, test engineers can opt to professional or freeware applications to automate their test case creation.

Wrong Application Usage
This is either using pair testing in wrong application, or using it in correct application but in its wrong area. Pair testing is not applicable to high-risk applications like medical systems. It should be used in collaboration with other testing procedures to ensure its reliability. This is because the high-risk systems encounter fatal errors from useless combinations being ignored by pair testing. Wrong usage also includes employing pair testing in areas where simpler methods with less effort are more applicable.

Wrong Parameter Selection
Success of pair testing depends on how well the parameter interaction is defined. Using wrong parameters in pairing will yield low success rate. Sometimes, choosing the best test parameters calls for involvement of requirements analyst, developer, and test engineers.

Stress Test: In this case we increase the load of the system to check if it can sustain such load and what is the bottleneck point. Also when system fails for such heavy load can it recover gracefully.

 

Load Test: In the tested system we will continuously increase the pressure until the performance index reached its limit, response time exceeds a predetermined target, or a resource has already reached saturation point. Load test can find system processing limit, to provide a basis for system tuning.

Large data volume test: In this case a large volume of data is used to test the system. This volume will be recorded in the requirement by the client, mainly the daily data volume transaction for the system. For example a system will process 700 transactions per day. Configuration test: Test to determine what is the optimal level of recourses needed by the system to run.

Reliability Test: You can increase CPU resources to maintain a 70% -90% utilization of the pressure. Then continue pressure on the system to run 8 hours, and do the analysis of the stability of the system based on the results. Concurrency Testing: Run same transactions simultaneously to find some algorithm design problems.

Performance testing using concurrent users:

1.    Performance testing is mainly to find software problems and hardware bottlenecks.

2.    At the requirement phase people should find all the performance requirements which are also called non functional requirements.

3.    For each performance requirements we should have one or two test cases.

4.    Is expected to test the performance of indicators are usually single-user oriented, if it involves concurrent users, then we need to design test cases accordingly.

Network Performance Testing

The main focus of this testing to determine the number of users and network bandwidth relationship. It also shows how the bandwidth, load and port change affects the user's response time. The best way is to adjust the performance of soft and hard combination.

Large data volume testing

Mainly aimed at the database systems have special requirements for testing, divided into three types:

1. Real-time large amounts of data: real-time simulation of a user working with large amounts of data, the main purpose was to test the user operations have a greater amount of data and the system can run stable.

2. Limit state test: The main focus is to test the system for a period of time where the system accumulate a certain amount of data and can normally run the business

3. The combination of the first two: Test the system has accumulated large amounts of data, some real-time transactions have a greater amount of data and can work steadily.

Stress Test: In this case we increase the load of the system to check if it can sustain such load and what is the bottleneck point. Also when system fails for such heavy load can it recover gracefully.

Load Test: In the tested system we will continuously increase the pressure until the performance index reached its limit, response time exceeds a predetermined target, or a resource has already reached saturation point. Load test can find system processing limit, to provide a basis for system tuning.

Large data volume test: In this case a large volume of data is used to test the system. This volume will be recorded in the requirement by the client, mainly the daily data volume transaction for the system. For example a system will process 700 transactions per day.
Configuration test: Test to determine what is the optimal level of recourses needed by the system to run.

Reliability Test: You can increase CPU resources to maintain a 70% -90% utilization of the pressure. Then continue pressure on the system to run 8 hours, and do the analysis of the stability of the system based on the results.
Concurrency Testing: Run same transactions simultaneously to find some algorithm design problems.

Performance testing using concurrent users:

1.    Performance testing is mainly to find software problems and hardware bottlenecks.

2.    At the requirement phase people should find all the performance requirements which are also called non functional requirements.

3.    For each performance requirements we should have one or two test cases.

4.    Is expected to test the performance of indicators are usually single-user oriented, if it involves concurrent users, then we need to design test cases accordingly.

Network Performance Testing

The main focus of this testing to determine the number of users and network bandwidth relationship. It also shows how the bandwidth, load and port change affects the user's response time. The best way is to adjust the performance of soft and hard combination.

Large data volume testing

Mainly aimed at the database systems have special requirements for testing, divided into three types:

1. Real-time large amounts of data: real-time simulation of a user working with large amounts of data, the main purpose was to test the user operations have a greater amount of data and the system can run stable.

2. Limit state test: The main focus is to test the system for a period of time where the system accumulate a certain amount of data and can normally run the business

3. The combination of the first two: Test the system has accumulated large amounts of data, some real-time transactions have a greater amount of data and can work steadily.

<!-- /* Style Definitions */ p.MsoNormal, li.MsoNormal, div.MsoNormal {mso-style-parent:""; margin:0in; margin-bottom:.0001pt; mso-pagination:widow-orphan; font-size:12.0pt; font-family:"Times New Roman"; mso-fareast-font-family:"Times New Roman";} @page Section1 {size:8.5in 11.0in; margin:1.0in 1.25in 1.0in 1.25in; mso-header-margin:.5in; mso-footer-margin:.5in; mso-paper-source:0;} div.Section1 {page:Section1;} -->

<!-- /* Style Definitions */ p.MsoNormal, li.MsoNormal, div.MsoNormal {mso-style-parent:""; margin:0in; margin-bottom:.0001pt; mso-pagination:widow-orphan; font-size:12.0pt; font-family:"Times New Roman"; mso-fareast-font-family:"Times New Roman";} @page Section1 {size:8.5in 11.0in; margin:1.0in 1.25in 1.0in 1.25in; mso-header-margin:.5in; mso-footer-margin:.5in; mso-paper-source:0;} div.Section1 {page:Section1;} /* List Definitions */ @list l0 {mso-list-id:1747993280; mso-list-type:hybrid; mso-list-template-ids:-289893936 67698703 67698713 67698715 67698703 67698713 67698715 67698703 67698713 67698715;} @list l0:level1 {mso-level-tab-stop:.5in; mso-level-number-position:left; text-indent:-.25in;} ol {margin-bottom:0in;} ul {margin-bottom:0in;} -->

WEB Performance Test Category

 

Stress Test: In this case we increase the load of the system to check if it can sustain such load and what is the bottleneck point. Also when system fails for such heavy load can it recover gracefully.

 

Load Test: In the tested system we will continuously increase the pressure until the performance index reached its limit, response time exceeds a predetermined target, or a resource has already reached saturation point. Load test can find system processing limit, to provide a basis for system tuning.

 

Large data volume test: In this case a large volume of data is used to test the system. This volume will be recorded in the requirement by the client, mainly the daily data volume transaction for the system. For example a system will process 700 transactions per day.

 

Configuration test: Test to determine what is the optimal level of recourses needed by the system to run.

 

Reliability Test: You can increase CPU resources to maintain a 70% -90% utilization of the pressure. Then continue pressure on the system to run 8 hours, and do the analysis of the stability of the system based on the results.

 

Concurrency Testing: Run same transactions simultaneously to find some algorithm design problems.

 

Performance testing using concurrent users:

 

1. Performance testing is mainly to find software problems and hardware bottlenecks.

 

1. At the requirement phase people should find all the performance requirements which are also called non functional requirements.

 

1. For each performance requirements we should have one or two test cases.

 

1. Is expected to test the performance of indicators are usually single-user oriented, if it involves concurrent users, then we need to design test cases accordingly.

 

Network Performance Testing

 

The main focus of this testing to determine the number of users and network bandwidth relationship. It also shows how the bandwidth, load and port change affects the user's response time. The best way is to adjust the performance of soft and hard combination.

 

Large data volume testing

 

Mainly aimed at the database systems have special requirements for testing, divided into three types:

 

1. Real-time large amounts of data: real-time simulation of a user working with large amounts of data, the main purpose was to test the user operations have a greater amount of data and the system can run stable.

 

2. Limit state test: The main focus is to test the system for a period of time where the system accumulate a certain amount of data and can normally run the business

 

3. The combination of the first two: Test the system has accumulated large amounts of data, some real-time transactions have a greater amount of data and can work steadily.

 

  To build a functional testing framework we need to define the following software testing artifacts at the very first stage of the project.

 

1. Test Plan: This needs to be written in a very early stage of the project (typically at the time of project design phase). The test manager needs to sit with all the project stakeholders and client to get it done, reviews and signed off.

Typically it should contain the following important things:

i. The total number of the resources (tester, test lead and test manager) needed to complete the project. So a clear stuffing plan must be determined.

ii. The test schedule is another important part of the test plan. Ideally it should not be a very tight. Put some buffer time in place. In many cases you will see delay in the original testing due to development delay or decision pending from client so if you do not put some buffer time those unseen risks and issues can not be handled and you end up doing lots of work to meet the schedule.

iii. Test environment configuration: Clearly define the hardware and software needs, their license procurement procedure etc. If you are working in offshore and onshore model then mention the access restrictions to the onshore systems from offshore if there is any, the performance issue, do you need extra bandwidth to access the onshore servers etc.

iv. Write down the items to be tested, items which are beyond the scope of the testing.

v. Clearly define a knowledge transfer plan for the system you are going to test. It may need you to send some tester to onsite to learn the system and understand the requirements.

vi. Identify the test exit criteria or when you can stop the testing. It varies from client to client. But in the SOW/contract document it should be clearly mentioned that when you can say the testing is done.

2. Test cases: The test case documents tell you how to do the testing for a functionality. Each requirement should have ideally 4 test cases associated with it viz two boundary values, one negative test and one happy path testing. It should have the test objective/business scenario, test steps which tells how to perform the testing, test data, expected result and actual result mentioned.

3. Defect Report guide: this is the most commonly used document for any functional testing project. Any deviation from the original requirement should be logged as a defect. A defect incident report should contain the following things in minimum:

    * Defect ID:  Defect ID number which is unique. No duplication of defect ID’s possible

    * Reporter’s/Originator Name on the bug. If there are questions we need to know who originated this report.
    * Assigned to (This bug is assigned); who’s going to be responsible for the bug and do the work on the bug?
    * Defect Status    State of the defect like submitted, assigned, planned, completed, re-tested, closed etc.,

    *  Defect Type  Defect Type may be logical, Data, Documentation, Bad fix, Third party, Performance etc.,

    * Date of origin  Date of origin of the defect like when it was found.

    * Build or Version number This gives the details of the code being worked on.
    * Phase found The defect found in (like Requirement, design, coding, testing) phase.
    * Origin of defect The defect originated in (like Requirement, design, coding, testing) phase.
    * Severity and Priority of the defect -  Severity (High, medium, low) and Priority (High, medium, low) of the bug

    * Brief Description of defect (what the problem is) For example, “Fatal error when printing landscape.” is a good description; short and to the point
    * Platform on which the bug found on
    * Windows, Linux, etc. Is the bug specific to one platform or does it occur on all platforms?
    * Feature or Specification or part of the code.
    * This facilitates assigning the bug to a developer assigned to that part of the product.
    * Test case No.   Test case reference number given here like Test_case v1.0 (where the defect is found)
    * Details including how to reproduce the bug and any other relevant data or clues about the bug.

-         Start with how the computer and software is setup.

-         List each and every step (don’t leave any out) to produce the bug.

-         Attach Snapshot of the error.

-         Sometimes a minor detail can make all the difference in duplicating or not duplicating a bug. For example, using the keyboard versus using the mouse may product very different results when duplicating a bug.

-         Attach the screenshot of the error if there is any

4. Test Summary Report: This contains the report and summary of the project test results, citing the relevant test data and statistics, and record the remaining issues. It typically contains total number of defects logged in the testing phase, the severity level of those defect, defects type and how many of them are still open or deferred to the next release.Also test exit report/summary report is an official sign off from the test team.

<!-- /* Font Definitions */ @font-face {font-family:Wingdings; panose-1:5 0 0 0 0 0 0 0 0 0; mso-font-charset:2; mso-generic-font-family:auto; mso-font-pitch:variable; mso-font-signature:0 268435456 0 0 -2147483648 0;} /* Style Definitions */ p.MsoNormal, li.MsoNormal, div.MsoNormal {mso-style-parent:""; margin:0in; margin-bottom:.0001pt; mso-pagination:widow-orphan; font-size:12.0pt; font-family:"Times New Roman"; mso-fareast-font-family:"Times New Roman";} p {mso-margin-top-alt:auto; margin-right:0in; mso-margin-bottom-alt:auto; margin-left:0in; mso-pagination:widow-orphan; font-size:12.0pt; font-family:"Times New Roman"; mso-fareast-font-family:"Times New Roman";} span.apple-converted-space {mso-style-name:apple-converted-space;} @page Section1 {size:8.5in 11.0in; margin:1.0in 1.25in 1.0in 1.25in; mso-header-margin:.5in; mso-footer-margin:.5in; mso-paper-source:0;} div.Section1 {page:Section1;} /* List Definitions */ @list l0 {mso-list-id:109592554; mso-list-type:hybrid; mso-list-template-ids:1457299780 1310989400 -886018122 -59082866 -434583240 232973728 -226603200 -898878016 1300807712 939422248;} @list l0:level1 {mso-level-number-format:bullet; mso-level-text:; mso-level-tab-stop:.5in; mso-level-number-position:left; text-indent:-.25in; font-family:Wingdings;} @list l0:level2 {mso-level-start-at:178; mso-level-number-format:bullet; mso-level-text:-; mso-level-tab-stop:1.0in; mso-level-number-position:left; text-indent:-.25in; font-family:"Times New Roman";} @list l1 {mso-list-id:647440276; mso-list-type:hybrid; mso-list-template-ids:662058904 1112860956 -1680573708 -2073791816 -2111792474 1286479260 1817996010 -290802148 -583658558 1022372298;} @list l1:level1 {mso-level-number-format:bullet; mso-level-text:; mso-level-tab-stop:.5in; mso-level-number-position:left; text-indent:-.25in; font-family:Wingdings;} @list l1:level2 {mso-level-start-at:178; mso-level-number-format:bullet; mso-level-text:-; mso-level-tab-stop:1.0in; mso-level-number-position:left; text-indent:-.25in; font-family:"Times New Roman";} @list l2 {mso-list-id:883979805; mso-list-type:hybrid; mso-list-template-ids:892864108 -1643090190 -1458635918 -607496470 -741162212 -1635090124 509744008 -1730219140 2104236370 -787186936;} @list l2:level1 {mso-level-number-format:bullet; mso-level-text:; mso-level-tab-stop:.5in; mso-level-number-position:left; text-indent:-.25in; font-family:Wingdings;} @list l3 {mso-list-id:941717932; mso-list-type:hybrid; mso-list-template-ids:782396364 -456476216 -1267981814 863554196 322087480 -652825166 316552290 272534206 -86745956 1361724468;} @list l3:level1 {mso-level-number-format:bullet; mso-level-text:; mso-level-tab-stop:.5in; mso-level-number-position:left; text-indent:-.25in; font-family:Wingdings;} @list l4 {mso-list-id:1469276748; mso-list-type:hybrid; mso-list-template-ids:-1628149048 215014748 -1634159382 -314939846 1547729894 -1345841828 -775681824 -209317054 379763420 1055534172;} @list l4:level1 {mso-level-number-format:bullet; mso-level-text:; mso-level-tab-stop:.5in; mso-level-number-position:left; text-indent:-.25in; font-family:Wingdings;} @list l5 {mso-list-id:1528175238; mso-list-type:hybrid; mso-list-template-ids:-1823419544 -370133166 -1882537810 -1916913256 220103274 -1446844612 1918824680 -1319174852 -1568005280 -1553821732;} @list l5:level1 {mso-level-number-format:bullet; mso-level-text:; mso-level-tab-stop:.5in; mso-level-number-position:left; text-indent:-.25in; font-family:Wingdings;} @list l5:level2 {mso-level-start-at:178; mso-level-number-format:bullet; mso-level-text:-; mso-level-tab-stop:1.0in; mso-level-number-position:left; text-indent:-.25in; font-family:"Times New Roman";} ol {margin-bottom:0in;} ul {margin-bottom:0in;} --> To build a functional testing framework we need to define the following software testing artifacts at the very first stage of the project.

1. Test Plan: This needs to be written in a very early stage of the project (typically at the time of project design phase). The test manager needs to sit with all the project stakeholders and client to get it done, reviews and signed off.

Typically it should contain the following important things:

i. The total number of the resources (tester, test lead and test manager) needed to complete the project. So a clear stuffing plan must be determined.

ii. The test schedule is another important part of the test plan. Ideally it should not be a very tight. Put some buffer time in place. In many cases you will see delay in the original testing due to development delay or decision pending from client so if you do not put some buffer time those unseen risks and issues can not be handled and you end up doing lots of work to meet the schedule.

iii. Test environment configuration: Clearly define the hardware and software needs, their license procurement procedure etc. If you are working in offshore and onshore model then mention the access restrictions to the onshore systems from offshore if there is any, the performance issue, do you need extra bandwidth to access the onshore servers etc.

iv. Write down the items to be tested, items which are beyond the scope of the testing.

v. Clearly define a knowledge transfer plan for the system you are going to test. It may need you to send some tester to onsite to learn the system and understand the requirements.

vi. Identify the test exit criteria or when you can stop the testing. It varies from client to client. But in the SOW/contract document it should be clearly mentioned that when you can say the testing is done.

2. Test cases: The test case documents tell you how to do the testing for a functionality. Each requirement should have ideally 4 test cases associated with it viz two boundary values, one negative test and one happy path testing. It should have the test objective/business scenario, test steps which tells how to perform the testing, test data, expected result and actual result mentioned.

3. Defect Report guide: this is the most commonly used document for any functional testing project. Any deviation from the original requirement should be logged as a defect. A defect incident report should contain the following things in minimum:

• Defect ID:  Defect ID number which is unique. No duplication of defect ID’s possible

• Reporter’s/Originator Name on the bug. If there are questions we need to know who originated this report.
• Assigned to (This bug is assigned); who’s going to be responsible for the bug and do the work on the bug?
• Defect Status    State of the defect like submitted, assigned, planned, completed, re-tested, closed etc.,

•  Defect Type  Defect Type may be logical, Data, Documentation, Bad fix, Third party, Performance etc.,

• Date of origin  Date of origin of the defect like when it was found.

• Build or Version number This gives the details of the code being worked on.
• Phase found The defect found in (like Requirement, design, coding, testing) phase.
• Origin of defect The defect originated in (like Requirement, design, coding, testing) phase.
• Severity and Priority of the defect -  Severity (High, medium, low) and Priority (High, medium, low) of the bug

• Brief Description of defect (what the problem is) For example, “Fatal error when printing landscape.” is a good description; short and to the point
• Platform on which the bug found on
• Windows, Linux, etc. Is the bug specific to one platform or does it occur on all platforms?
• Feature or Specification or part of the code.
• This facilitates assigning the bug to a developer assigned to that part of the product.
• Test case No.   Test case reference number given here like Test_case v1.0 (where the defect is found)
• Details including how to reproduce the bug and any other relevant data or clues about the bug.

-         Start with how the computer and software is setup.

-         List each and every step (don’t leave any out) to produce the bug.

-         Attach Snapshot of the error.

-         Sometimes a minor detail can make all the difference in duplicating or not duplicating a bug. For example, using the keyboard versus using the mouse may product very different results when duplicating a bug.

-         Attach the screenshot of the error if there is any

4. Test Summary Report: This contains the report and summary of the project test results, citing the relevant test data and statistics, and record the remaining issues. It typically contains total number of defects logged in the testing phase, the severity level of those defect, defects type and how many of them are still open or deferred to the next release.Also test exit report/summary report is an official sign off from the test team.

Certified Software Test Manager(CSTM) certification is provided by STQC IT Service Centre, India.

About CSTM Certification:

• CSTM certification is a formal recognition of competenceof practicing professionals
• It is the process of validation ad recognition of knowledge, skills and personal attributes among the testing professionals
• Professionals need to demonstrate their proficiency within a specified BOK of CSTM.

The Objectives of CSTM certification are as follows:

• Enable software professionals to acquire skills of software testing through training
• Enable them to become eligible for CSTM certification examinations
• Help them to acheive professional certification.
• Enable them to implement the knowledge and skills on software testing (Verification & Validation techniques) in day to day jobs.

People with the following job profiles will benefit from this course / certification:

• Software Testing Professionals
• Test Leads/Managers
• Members of Software Quality Assurance Group (SQA)
• Members of Software Engineering Process Group (SEPG)

The Eligibility Criteria for CSTM Certification is as follows:

• Bachelors degree in engineering or Master degree in Science / Computer Science with at least 2 years of relevant work experience
•  OR Diploma in Engineering or Bachelors degree with at least 3 years of relevant work experience
• The work experience should be relevant to the title and should be in the field of software
  development, inspection, reviews, testing and/or QA.

Prerequisites for CSTM Certification:

Must have attended the CSTM refresher course conducted at any STQC IT Service Centre.

Body of knowledge:

1. CSTM - An Introduction                    [General Information; no Questions]
2. Software Testing Introduction
3. Testing Standards & best practices    [25 Marks] i.e. 25 one mark questions
4. Static Testing                                 [15 Marks] i.e. 15 one mark questions
5. Dynamic Testing                             [30(5+25) Marks] i.e. 5 one mark questions and 5 descriptive type questions
6. Testing throughout the life cycle       [30 Marks] i.e. 30 one mark questions
7. Test Management                           [50(25+25) Marks] i.e. 25 one mark questions and 5 descriptive type questions

The above information is also available in CSTM Course Material. Please refer the course material for any future updation.

Course Contents:

The body of knowledge for CSTM contains below essential knowledge areas of software development lifecycle. These modules are available in CSTM course book Version 5:

• CSTM Introduction
• Principles of Testing
• Testing Standards ad best practices
• Static Testing
• Dynamic Testing
• Testing throughout the life cycle
• Test Management 

Participants (scoring >=60% marks) will be awarded "Certificate of Achievement".

The CSTM certification will make you confident to undertake test management activities and a professional pride in belonging to the School of Certified Professionals.

For further detailed information of the Certification please check the following link:
http://www.stqc.nic.in/searchdetail.asp?lid=660&skey=cstm

Certified Software Test Manager(CSTM) certification is provided by STQC IT Service Centre, India.

About CSTM Certification:

• CSTM certification is a formal recognition of competenceof practicing professionals
• It is the process of validation ad recognition of knowledge, skills and personal attributes among the testing professionals
• Professionals need to demonstrate their proficiency within a specified BOK of CSTM.

The Objectives of CSTM certification are as follows:

• Enable software professionals to acquire skills of software testing through training
• Enable them to become eligible for CSTM certification examinations
• Help them to acheive professional certification.
• Enable them to implement the knowledge and skills on software testing (Verification & Validation techniques) in day to day jobs.

People with the following job profiles will benefit from this course / certification:

• Software Testing Professionals
• Test Leads/Managers
• Members of Software Quality Assurance Group (SQA)
• Members of Software Engineering Process Group (SEPG)

The Eligibility Criteria for CSTM Certification is as follows:

• Bachelors degree in engineering or Master degree in Science / Computer Science with at least 2 years of relevant work experience
•  OR Diploma in Engineering or Bachelors degree with at least 3 years of relevant work experience
• The work experience should be relevant to the title and should be in the field of software
  development, inspection, reviews, testing and/or QA.

Prerequisites for CSTM Certification:

Must have attended the CSTM refresher course conducted at any STQC IT Service Centre.

Body of knowledge:

1. CSTM - An Introduction                    [General Information; no Questions]
2. Software Testing Introduction
3. Testing Standards & best practices    [25 Marks] i.e. 25 one mark questions
4. Static Testing                                 [15 Marks] i.e. 15 one mark questions
5. Dynamic Testing                             [30(5+25) Marks] i.e. 5 one mark questions and 5 descriptive type questions
6. Testing throughout the life cycle       [30 Marks] i.e. 30 one mark questions
7. Test Management                           [50(25+25) Marks] i.e. 25 one mark questions and 5 descriptive type questions

The above information is also available in CSTM Course Material. Please refer the course material for any future updation.

Course Contents:

The body of knowledge for CSTM contains below essential knowledge areas of software development lifecycle. These modules are available in CSTM course book Version 5:

• CSTM Introduction
• Principles of Testing
• Testing Standards ad best practices
• Static Testing
• Dynamic Testing
• Testing throughout the life cycle
• Test Management 

Participants (scoring >=60% marks) will be awarded "Certificate of Achievement".

The CSTM certification will make you confident to undertake test management activities and a professional pride in belonging to the School of Certified Professionals.

For further detailed information of the Certification please check the following link:
http://www.stqc.nic.in/searchdetail.asp?lid=660&skey=cstm