Software Testing Social Network

There is a strong relationship between the number of test cases and the number of function points.  As expected there is a strong relationship between the number of defects and the number of test cases and number of function points.

{mosgoogle left} 

The number of acceptance test cases can be estimated by multiplying the number of function points by 1.2.  Like function points, acceptance test cases should be independent of technology and implementation techniques.

For example, if a software project was 100 function points the estimated number of test cases would be 120.  To estimate the number of potential defects is more involved.  

Estimating Defects

Intuitively the number of maximum potential defects is equal to the number of acceptance test cases which is 1.2 x Function Points.

Preventing, Discovering and Removing Defects

To reduce the number of defects delivered with a software project an organization can engage in a variety of activities.  While defect prevention is much more effective and efficient in reducing the number of defects, most organization conduct defect discovery and removal.  Discovering and removing defects is an expensive and inefficient process.  It is much more efficient for an organization to conduct activities that prevent defects.

Defect Removal Efficiency

If an organization has no defect prevention methods in place then they are totally reliant on defect removal efficiency.

1. Requirements Reviews up to 15% removal of potential defects
2. Design Reviews up to 30% removal of potential defects
3. Code Reviews up to 20% removal of potential defects
4. Formal Testing up to 25% removal of potential defects 

{mosgoogle left} 

In other words, if your organization is great at defect removal the maximum percentage of defects your organization can expect to remove is 90%.  If a software project is 100 function points, the total number of maximum (or potential) defects could be 120.  If you were perfect at defect removal your project would still have up to 12 defects after all your defect discovery and removal efforts.  The far majority of organization would receive a B (medium) or even a D (poor) at defect removal efficiency. 

Defect Discovery and Removal

An organization with a project of 2,500 function points and was about medium at defect discovery and removal would have 1,650 defects remaining after all defect removal and discovery activities.  The calculation is 2,500 x 1.2 = 3,000 potential defects.  The organization would be able to remove about 45% of the defects or 1,350 defects.  The total potential defects (3,000) less the removed defects (1,350) equals the remaining defects of 1,650.

Defect Prevention

If an organization concentrates on defect prevention (instead of defect detection) then the number of defects inserted or created is much less.  The amount of time and effort required to discover and remove this defects is much less also.

1. Roles and Responsibilities Clearly Defined up to 15% reduction in number of defects created
2. Formalized Procedures up to 25% reduction in number of defects created
3. Repeatable Processes up to 35% reduction in number of defects created
4. Controls and Measures in place up to 30% reduction in number of defects created

Imagine an organization with items 1 and 2 in place.  A project with 100 function points would have a potential of 120 defects, but since they have preventative measures in place, they can reduce the number of potential defects by 48 (40% = 25% + 15%).  That makes the potential number of defects 72 compared to 120 with no preventative efforts.  Assuming that an organization was medium at defect discovery and removal they could remove 45% of the remaining defects or have 40 remaining when the project rolled to production.

The above table represents the number of defects that an organization that does items 1 and 2 above and is medium at discovery and removal.

The problem for estimating defects is multidimensional.  First the total number of defects must be estimated.  Second the impact of defect prevention needs to be understood and the estimated number of defects adjusted.  Third an assessment needs to be done to understand how many defects can be discovered and removed by an organization.

Clearly, the fewer number of defects that an organization must discover and remove the better.  The way this is accomplished is by better process, a more stable organization and repeatable processes.  The focus of software organizations needs to be on defect prevention instead of defect detection.

Comments (0)

Subscribe to this comment's feed