Estimation technique- Test Point Analysis

Estimation is a management activity that is often a predecessor to planning. Estimation creates approximate cost and schedule targets for tasks like test execution or automation creation.  There are multiple factors affecting the test estimation as mentioned here. In Test estimation, we need to apply known estimation best practices to the testing work throughout the lifecycle. Major activities which occur during a test project are listed below – 

1. Planning
2. Control
3. Analysis
4. Design
5. Implementation
6. Execution
7. Reporting

Out of the above test, execution is the one that gets the highest management attention. Test execution is a critical path in terms of overall project management as well. However, it is known that test execution estimates are troublesome as it is dependent on various dynamic factors like environment availability, Integration issues, data availability, Defect fix rate, etc.  Hence, the determining factor for test execution is not only how long it takes to run each test case once but also the time required to find, fix and retest the fixed critical bugs.
Please note, there will always be a set of assumptions during estimations. As a leader, you must ensure that all assumptions are documented and monitored during test control.

There are various techniques for test estimations which can be broadly categorized into two categories-

1. Metrics based techniques
2. Experience-based techniques.

Metric based Estimation Techniques: 
As the name suggests, It involves estimating the testing effort based on metrics of former or similar projects or based on typical values. 
Various techniques under this broad category include:

1. Work Breakdown Structure
2. Test Point analysis.
3. Percentage of overall project effort
4. Team Estimation – Example Three-point estimation or Delphic oracle

Expert Based approach:
It involves estimating the tasks based on estimates made by the owner of the tasks or by experts of the system or the skill. 

Test Point Analysis (TPA) : 
Test point analysis is a technique used to estimate black box testing. It corresponds to Function point analysis which is used for White box testing estimations. It makes use of 3 entities to make an estimate, namely –

1. Size – 
   This is determined in terms of functional points adjusted for complexity, interfaces, and uniformity.
2. Strategy –
   In particular which quality attributes or risks are to be tested and to what extent. 
3. Productivity –
   This gives us productivity levels of participants, which is determined by the skills of the testing team and are influenced by the process followed by organization, technology being used.

How to calculate Test Points :
1. First we calculate, For each functional area of the system, a value that gives the effort weightage for the dynamic quality characteristics i.e Quality characteristics which will be tested with dynamic tests. (Actual code gets executed in dynamic tests)

• Four characteristics are considered – Functionality, Usability, Security and Efficiency.
• A weight is given using numbers 0,3,4,5,6 depending upon the importance of dynamic testing for each quality characteristic.
• Smaller numbers indicate less importance, large numbers indicate more importance.
• Default value for weight is 4, indicated in bold. 

Dynamic Quality Characteristics

Qd = (.75 Qf+0.05Qp+0.1 Qu +0.1 Qe)/4
Qf= Functionality Factor
Qp- Security Factor
Qu= Usability Factor
Qe= Efficiency Factor

2. Secondly, we calculate for each functional area of the system, a function-dependent weighing factor based on various attributes that will make it easier or harder to test.

• Five Factors considered are – Usage Importance, Usage Intensity, Interfacing, Complexity, and Uniformity.
• A weightage number is given for each factor as given in the figure.
• The default value for each attribute is highlighted in bold. 
• Lower numbers indicate it is easy or less critical to test.
• For uniformity, the smaller number indicates harder to test. 

Function Dependent Factors

Df = ((Ue+Uy+I+C))20)*U
Ue= Usage Importance. 3,6 or 12
Uy= Usage Intensity, 2,4,or 12
I= Interfacing; 2,4, or 6
C= Complexity 3,6, or 12
Uniformity 0.6 or 1

3. Third calculation is dynamic test point calculation for a given functional area of the system. This could be a lengthy process if the system is complex.

• Multiply the dynamic quality weighting factor with a function-dependent weighting factor and function point count of the given functional area.
• This process is repeated to calculate dynamic test point count for all functional areas of the entire system.

Dynamic test Points

TPf= Qd*Df*FPf
Pf= Function point count for function under consideration

4. Fourth calculation is Static test points i.e the points which we intend to cover by static testing of the system (without code execution).

• 16 points are given for each of 6 ISO 9126 Quality characteristics that will be addressed by Static analysis . Ex- Code maintainability etc.
• Static test points can range from 0-96

Qs= SumQi
Qi= 16 per statically tested quality characteristics

5. Lastly, we calculate the total test points. 

• Sum all the dynamic test points across all functional areas
• Multiply the function point count for the entire system by static test points
• Divide the product by 500.

TP= Sum(TPf)+(FP*Qs)/500

How to calculate Test Hours using Test Points :
1. Productivity factor –

• It ranges from 0.7 to 2.0
• Smaller Numbers mean more productive, larger number means less productive
• This is usually calculated from historical data i.e You will need to calculate the test point and compare them to test hours for two or three previous projects to calibrate the value for productivity factor.

2. Environmental Weighting factor –

• Rate the situation with test tools, development testing, test basis test environment, development environment and testware as underlying attributes
• Smaller number better environment for testing, larger number means worse environment which can cause inefficient testing
• Default values are shown in bold.

E= Tt+Dt+Tb+De+Te+Tw
Tt= Test Tools 1,2 or 4 
Dt= Development Testing 2, 4 or 8
Tb=  Test Basis, 3 ,6 or 12 
De= Development environment 3,4, or 8 
Te= Test Environment 1,2 or 4
Tw= Testware 1,2 or 4

3. Primary test Hours –

• Multiple test points by productivity factor by an environmental factor

PT = P*E*TP
TP= Test points from previous steps.

4. Management Overhead –

• a. Consider Team size and management tools.
• b. Bigger numbers, as obvious, implied more overhead, and hence more total hours.
• c. Default values for factors are given in bold. 

MO= 1+(Ts+Mt)/100
Ts= team Size
Mt= management Tools

5. Total Hours –

• Total hours are calculated by multiplying primary test hours by management overhead 
• Test hours are then distributed across the test process explained in the beginning as per percentages shown below.

TH= PT*MO

Test Hours Breakdown : 

1. 10% Preparation (Plan, design)
2. 40% Specification (Test Dev)
3. 45% Execution (Run, Report)
4. 5% Completion (Final reports)

Closing Note :
It is obviously a complex model for estimation. Some practical and important factors like outsourcing are not considered in this. However, this model gives a good idea of how estimation can be built. This can be tailored for the organization or the project. Given we have good historical data available, a very accurate estimation can be derived from this model.