Software Testing Social Network

The basic purpose of this document is to give a high level introduction to Software Load and Performance testing methodologies and strategies. This document is intended to facilitate Software test Managers, Project Managers, Software Engineers, Test Leads, Test engineers, and QA leads — anyone who is responsible for planning and/or implementation of a successful and cost effective performance testing program.
 

Scope

The scope of all the conceptualization mentioned in this document is only in the Test Execution in Automation Context.

In this context the attributes of Load & Performance testing covered are as follows,

Load / Performance Test Planning

Load / Performance Tool Evaluation / Selection

Load Test Process / Methodology and Test Strategy

Load / Performance Test Start/Stop Criteria

Test Environmental Setup and Pre-Requisites

Test Scenarios Definition including Load Scenario, Data Volume, Virtual Users Ramp Rates and Scripting Guidelines etc.

Pass / Fail / Exit Criteria

Analysis and Report Generation
{mosgoogle left} 
 
2       Load / Performance Test Planning 

To make any operation/mission successive planning plays the most vital role and according to the 80-20 theory, 80% of the time should be spent in planning and 20% in real time plan execution/implementation. In the similar fashion, Software performance test planning is very crucial as well.
Any Software Performance Test Plan should have the minimal contents such as,
Performance Test Strategy and Scope Definitions.

-Test Process and Methodologies to follow in different test phases.

-Test Tool Details (Tool Evaluation, Selection, Configuration, Addins, Third Party Utilities Integration, Os Configurations etc)

-Test Cases Details including Scripting, Library Development and Script Maintenance Mechanisms to be used in Every Performance Test Phases.

-Resource Allocations and Responsibilities for Test Participants.

-Test Life Cycle Tasks Management and Communication Media.

-Risk Management Definitions.

-Test Start /Stop Criteria along with Pass/Fail Criteria Definitions.

-Test Environment Setup Requirements. (Hardware Setup, Software Setup, Os Configurations, Third Party Utilities Setup etc.)

-Multi-Platform Performance Test Strategies.

-Application Deployment Setup Requirements for Load Test.

-Virtual Users, Load (Iterations Vs Users), Volume Load Definitions for Different Load/Performance Test Phases.

-Results Analysis Algorithms and Reporting Format Definitions.

 

3       Load / Performance Tool Evaluation & Selection
Tool Evaluation is yet another important task in Performance Test Automation wherein there are several things to be considered. In cases wherein the tool evaluation and selection is completely done by the Client then readers may skip this Topic and proceed to next. 

While selecting any tool for Load or/and Performance testing the following things should be analyzed such as, 

-Test Budget Vs Available Load/Performance tools in the market mapping to the Budget.

-Protocols, Development & Deployment Technologies, Platforms, Middle-Ware tools / Servers, Third-Party Integrations of the Application Under test Vs Support for these factors in the available tools with prioritization of the availability in the tool for the Scope of expected test.

-Complexity of the Tool Usage Vs Availability of the tool experts along with the timeline requirements for the tool scripting / load scenario creation / tool configuration with respect to Man-hours and Other Resource Requirements.

-Tools Limitations and Work-Around factor mapping with the current scope of testing.

-ToolÙs Integration / Portability Factors with Other Tools used to Monitoring, Analyzing and Test Management.

-On Evaluation and Selection of Base tool, third party monitors / tools to be used in Integration with Main Load Testing Tool should be defined.

(Third Party Monitors / Tools like ‘Optimize ITÙ, ‘WeblogicÙ, ‘Oracle ToolsÙ, ‘Spotlight On OracleÙ, ‘Fog-LightÙ and Test Management Tools like Mercury-Test Director, Segue-Silk Plan Pro, Compuare-QA Center etc.)
  

4       Test Process / Methodology and Test Strategy 

Test Process, Methodology and Strategy for any Load / Performance testing will definitely vary for every project based on the Client Requirements / Company Process Implementations but still here we are going to put some light on the common generic performance test process, methodologies and strategies followed in the industry.

By using a methodology a project can make sure that resources are applied to the problem effectively and that the people involved approach the work in a structured way. In Performance testing Process there are couple of classifications as mentioned below,
Benchmark Testing

This task tests the newly coded performance features to assert that they do actually improve the performance of the application. There is always a danger that a new performance feature decreases the performance of the application instead of increasing it. Tests should be constructed so that they executed a standard benchmark style test with the feature switched on and with the feature switched off.

      The first time this particular task is performed the task becomes developing the benchmark tests. When this task is performed in later iterations it becomes re-running the benchmark tests with the new version of the code in order to track whether or not the code is improving.

 
Analysis Testing

      Benchmark tests are good for a basic understanding of what is happening and are very useful for tracking improvements over time but they are not so good at isolating the reasons of the next major performance problem. Analysis Testing refers to designing tests that attempt to isolate the next major performance problem. These kinds of tests may do something completely different to the benchmark tests in order to explore what is happening in the target application. The tests are designed to explore theories as to where the next performance problem may be.

      This is also where supplementary tools are used most often. Additional tools such as method-level profilers and operating system performance monitors can be useful in working out where a problem might be occurring.