Software Testing Social Network

“Scenario” is already used in the software industry since 1980s. However, it was first used by the United States in military planning in 1950s. In software development, scenarios are used in verifying suitability of a certain software application for its intended use through hypothetical stories.

Despite there is no procedural attempt to clearly establish software testing in a realistic environment, scenarios provide practical situations and user-oriented actions in validating software operations. It also allows testers to thoroughly understand the complex problems and systems behind the software. There are two types of scenario testing. First is field test, and the other is laboratory test.

Field testing is considered to be the least obtrusive because actual operating environment and user actions are being used, or at least mimicked, during testing. Actual factors like noise level, room capacity, amount of data processing, etc. should be used in testing. Work pace is also important because it can present other considerations like operation and idle time.

In laboratory testing, abnormal situations are being considered by having isolated cases of user actions. It does not need to be similar with day-to-day software usage but should be more focused in acquiring diverse and abnormal results using varying data collection methodologies. Because of this, lacking error handling capability is often detected in laboratory testing.

Between the two, field testing is more realistic and can be thought as “acceptance testing.” On the other hand, laboratory testing is more theoretical and can be thought as “fail-safe testing.” Users can benefit more from reliable field test while developers can benefit more from efficient laboratory test.

The characteristics of scenario testing are the following:
•    Story-based
•    Motivating
•    Credible
•    Complex Software Usage Environment
•    Testable
Scenarios should include stories, or sample situations on how the system will be used, and some information about the user. It should also be motivating to drive the persons involved in the development to fix the problem. One way of motivating them is to have credible story. It should be believable enough that such situations can occur in real-life and not a mere possibility. High software usage should also be considered to unlock hidden bugs and relationships between system features and lastly, the scenario should be testable.

Scenarios are quite different compared with other testing because a good scenario requires creativity and imagination. Thus, many teams are hesitant to implement it because they only see this difficulty and not its advantages. Listed below are the benefits of scenario testing:

Faster Learning of the System
Scenarios allow testers to investigate the entire system by themselves without following any guidelines or checklists. This causes them to understand the system faster because people can learn better by doing actual tasks their own way instead of using the system against its manual. This can also result to mastery and expert usage of the system.

Reveal Hidden Failures
Abnormal scenarios lead to exposing bugs caused when software has insufficient error handling. Unexpected user actions are also trapped by well-built scenarios.

Relate User Requirements with Software Operations
Properly defined field tests result to highly acceptable software. This is because normal scenarios represent user requirements and satisfying the scenario is satisfying the requirements.

Five keys to Choose the most effective Regression Test Cases

By definition regression testing is a new round of software validation after a build release with bug fixes.  According to Microsoft's statistics, in accordance with their experience, most developers will introduce one fresh defect after solving 3 ~ 4 defects. So we need to do regression testing to find those newly introduced bugs.
In general, higher the coverage of the regression, lower the risk, but the time it will take will be more and vice versa. So, if time allows, once should cover all test cases as part of the regression test suite but generally will not have so much time. This requires us to make a balance between effort it takes and coverage of the test use cases used as regression testing.

When choosing regression testing, the first thing to determine is the ratio of regression test cases, this situation should be based on time, and 100% are the best, but because of the time constrains this ratio is generally around 60%. Then we have to determine the regression test cases' priority.
Let's check the seven most common Regression test selection method:

1. First check the newly modified features (if there is any) in the new build release.

2. Then find the impact areas, meaning because of the introduced new features. What the closely coupled areas can get impacted? All those related modules need to be retested as part of regression testing.
3.  Include the main flows or highly used areas of the program. You can easily get the frequency of the use of a particular module, and if it is very high then that is the area you need to retest.

4. Furthermore, the most vulnerable parts of the program, for instance, security risks, data leakage, encryption registration;

5. If the above done, there is still time, then best to include some of the alternative flows test cases found the use case. Alternative flows are not happy path testing but some other ways to use the programs.

These are the regression test case selection priority. In most of the organization people use some automation tools to automate the regression test cases. It always has got good ROI.

Boundary analysis is a software testing technique that divides expected input data into three, first is those that belong within an expected range, second are those below the minimum value, and third are those above the maximum value. It is actually a variation of data partitioning known as equivalence partitioning.

Studies show that probability of encountering software fault increases as it approaches its boundary. Thus, this testing used equivalence partitioning and its three segments in testing. This testing is also founded from the assumption that if the system will yield the correct results from single representatives from each of the three segments, it will be true for all values that belong to each segment.

Boundary errors usually occur because of faulty logical conditions like greater than, less than, etc. Zero starting value for computer counting also contributes to incorrect developer logic.

The purpose of boundary analysis is to focus the test cases in areas most susceptible to errors through accurate pinpointing of minimum and maximum values. For example, text box in a software interface can accept numeric inputs between 1 to 100. Possible equivalence partition values can be 0, 5, and 101.

In performing boundary value analysis, there are two main steps to follow, identifying equivalence partition and designing test case.

Identifying Equivalence Partition
This is very similar with determining the minimum and maximum values then specifying values three values to satisfy the three equivalence partition. The difference is that expected output from user requirements are now included in consideration. For example, inputs above the maximum acceptable value will display certain information like items n the entire inventory, or anything that is written in the specifications.

Designing Test Case
Steps are the same with equivalence partitioning. However, rules in formulating their test cases are different. Instead of focusing on the equivalence values, the following rules should be applied:
Valid test item conditions should be written for value within the range, and invalid conditions should be written for values outside the range. Valid and invalid conditions can be combined depending on the style of the test case developers.
First and last entries of used files should be included in the consideration.
Search for other exaggerated input and output values then, write their corresponding test cases.

Just like with other testing methodologies, boundary analysis has its limitations. One is its inapplicability to some programming language. Strong-typed languages, or those that require a specific data type for each variable, has lower boundary analysis efficiency compared to be weak-type, or those that allow cross-typing of variables. However, modern technology already resolved this limitation by more advanced partitioning techniques. Another boundary value analysis limitation is its poor judgment when it comes to relationships between variables. This analysis can only cover expected behavior when a value fluctuates beyond its normal range. However, not when variable dependencies exist in the system.

Evident advantages of boundary value analysis are improving code robustness and preparing the system for worst-case scenarios. Robustness is improved because “clean” and “dirty” test cases are being utilized in testing. “Clean” cases represent those within the allowable range while “dirty” cases represent those outside the range. In addition, clean and dirty cases help in assessing the system capability to handle worst-case conditions.