Software Testing – Estimation Techniques

Test estimation is an important part of a project since test estimation plays a key role in test planning, scheduling project, and allocation of necessary resources. The following topics will be discussed here:

1. What is Software Test Estimation?
2. Why Test Estimation?
3. What To Estimate?
4. How To Estimate? 
5. Test Estimation Best Practices
6. Other Techniques

Let’s start discussing each of these topics in detail.

What is Software Test Estimation?

Test estimation comprises efforts contributed by Test Managers or Test Engineers for the completion of testing. Software test estimation starts with the breakdown of work using the fundamental process that includes the identification of various stages, activities, and tasks required to be performed. The set of tasks to be performed are as follows:

• Planning of the tests and controlling their execution.
• Design of the scenarios and Analysis.
• Implementation of the tests and execution.
• Evaluation of exit criteria.
• Test Closure and final sign-off.

Identification of activities for each phase is done and then the tasks and subtasks are defined. After that, we estimate how much time these tasks and subtasks will take to be completed.

Why Test Estimation?

• Test estimation is done to answer two critical questions, overall cost and time taken. This is an essential aspect for small projects that have a limited budget. 
• Test estimation basically acts as a forecast that helps maintain budget and time constraints. 
• Estimation is used to predict the efforts required and the time and cost that would be involved in completing the estimated tasks. As soon as the QA team is able to estimate the problem scenario, they can drive the situation effectively and reduce the overall time and budget constraints. 
• Test estimation involves experienced people in its core decision-making, they are responsible for implementing best QA practices like test case points, use case point methods, etc.

What to Estimate?

• Time: Time is one of the most critical aspects when it comes to projects as we have deadlines to meet and have SLAs in place for the delivery of projects on time.
• Resources: Resources (people, equipment, facilities) are required for carrying out project activities. Proper utilization of resources plays a key part in optimizing available resources and estimating the timelines.
• Cost: Cost refers to the project budget where resources and how much cost allocation is required for all the projects are defined.
• Skillset: Skillset refers to the knowledge and the experience available among the Team members. A person having a better skillset will take less time to cover the task compared to a person having a lower skillset. This is a crucial aspect when it comes to estimation
• Risks involved in the software and the project.

How To Estimate?

There are 4 stages to do software test estimation:

1. Divide the whole project into small tasks: In this step Work breakdown structure (WBS) technique can be used to divide a complete project into modules. The modules are divided into sub-modules. Each sub-modules is divided into functionality, thus the whole project is divided into the smallest tasks.

Work Breakdown Structure (WBS): This technique is used when the project is very large and has complex scenarios. Generally, this technique is often used when the testing team is not very sure about the test estimates. In this structure, we divide the project into simpler, and independent tasks. This is a top-down approach.

The approach of WBS:

• Developing a goal.
• Breaking the goal into its deliverables.
• Breaking the deliverables into activities.
• Further breaking the work into small and manageable components thereby allowing more time for resources.

2. Allocate each task to team members: In this step, each task is assigned to the appropriate team member in the team. 

3. Estimate the effort required to complete each task: Two techniques can be used to estimate the effort for tasks. Below are the two software test estimation techniques, Three-point estimation and Function point analysis.

Three Point Estimation: Three-point estimation is a technique that makes use of three types of costs or their duration estimates. They are as follows:

• Optimistic (O): Best case estimation is estimated with the assumption that the effort is getting channeled correctly and nothing goes wrong. All the scenarios are positive.
• Most Likely (M): Most likely estimation is estimated considering most of the things are going well but a problem could resurface in the future. Estimating is considered both positive and negative for specific scenarios.
• Worst case (W): Worst case estimation is estimated considering everything goes wrong. Estimation is considered negative for all scenarios.

This technique is used for improving the accuracy of the estimates of cost or duration involved. Here, each task is broken down into minor sub-tasks, and then the results are estimated. 

The formula to calculate estimation is:

Test estimation = (O + (4 * M) + W) / 6

Here,

• O: Estimation for best case scenarios.
• M: Estimation for most likely case scenarios.
• W: Estimation for worst-case scenarios.

The final result is calculated once we estimate all the case scenarios.

Example: Let us assume there is a requirement for testing the entire software application. The application has a lot of parts – Frontend, Backend, Databases, APIs, and performance testing. Now, estimating the possible scenarios:

• Scenario 1: Team ‘A’ requires 30 man-hours to test the entire application in the best-case scenario(O) i.e. a scenario where no problem arises, everything works as expected and the resources in the team were on the same page in meeting the deadlines and no backlogs were present in the sprint sessions of the SDLC phase.
• Scenario 2: Team ‘A‘ requires 45 man-hours to test the entire application in the most-likely scenario (M) i.e. a normal scenario where problems are expected to arise and most of the tasks would be completed with fewer backlogs in the upcoming sprints.
• Scenario 3: Team ‘A‘ requires 60 man-hours to test the entire application in the worst-case scenario (W) i.e. a scenario where almost the entire situation goes wrong due to a lack of skilled resources or due to the fact that many of the members are newly deployed to a team.

We have three values now:

O = 30
M = 45
W = 60

The average value for the test estimation (E) can be calculated using the formula:

E =  (O + (4 * M) + W) / 6
   = (30 + 4 * 45 + 60) / 6
   = 45 man hours

Standard deviation (SD) = (W – O) / 6
                                          = (60 – 30) / 6
                                          = 5

Hence, the final estimate would be: Team ‘A’ needs 45 +/- 5 person-hours to complete the testing of the application.

Function Point Estimation: Function Point Estimation begins with identifying the functions in the software. Once the software function to be tested is identified, it is distributed among the resources available, and the time required to test each function is calculated. 

In this estimation technique, we use weights and hence calculate the estimation for entire functions that are under test by multiplying the (weight * duration). To do this, we do the following steps:

• Identify the smallest function under test and assign the weight as One.
• Finding the time required to test the function.
• Identify other functions as well and similarly calculate the time to test the functions by multiplying the (weight * duration).
• Once all functions are estimated, we add them to calculate the total effort required for conducting the testing.

This technique introduces us to the concept of functional point (FP). The FP is assigned for the modules having varying complexities – Easy, medium, and hard. The tasks which are simpler as assigned lesser points and for difficult tasks higher number of points are assigned.

Example: Let us then take the example of the same application that we took under consideration for Three-Point Estimation. The application has various components: Frontend, Backend, database, and business logic layer. Now, all of the components in the application have easy, medium, and difficult modules. Therefore, segregating the project into modules: 

Assume FP = $10 per points.

Type of module	No: of modules	FP (for each module)	Total FP (for each module)
Easy	4	2	8
Medium	8	3	24
Difficult	3	5	15

Total FP for project = 4 * 2+ 8 * 3 +3 * 5 
                              = 47

Therefore, to complete the project we need 470$

4. Validate the estimation: Forward the estimation to the management board, who will review and approve it.

Test Estimation Best Practices

1. Resource planning in estimation: Resource planning and allocation play a key part in estimating the delivery of the project and meeting deadlines.  The availability of resources will not only deliver updates to key stakeholders but also assure that we set our estimations right.
2. Taking references from past projects: Some projects may resemble each other with respect to their domain testing or open-source frameworks or the same client. Any difficulties faced in the past should be noted and care must be taken so that they do not recur in the future.
3. Keeping some buffer time: Unforeseen circumstances can happen which include the departure of critical team members or unplanned long leaves. This could delay the scheduled timeline of the project and care must be taken to preserve some buffer time.
4. Sticking to the Estimation: Whatever estimation we have made, we should try to adhere to that irrespective of its correctness or feasibility. Unless any major change is encountered, we should try not to modify the estimate and stick to it.
5. Considering the Bug Cycle: The test estimation has the bug cycle included in it. The encountering of bugs causes a delay in the actual test cycle and hence the number of estimated days increases.
6. Scope of Project: Knowing our project objective is important as it helps to estimate between small and large projects and our project delivery depends on it.  Large projects have lots of test scripts, test data, and test documents. Therefore, knowing our objective of testing and planning of deliverables plays a key role in knowing the project scope.
7. Load Testing Plan: We need to know the estimated time if there is a requirement to perform Load Testing and hence plan the estimates accordingly.
8. Parallel Testing: The scope of parallel testing needs to be known i.e. whether we have the previous versions of the same product for comparing the output. If there is a chance, the testing process becomes easier and product estimation can be done in a better way. 

Other Techniques:

Some other techniques used are:

1. Wideband Delphi Technique: This is a technique where experienced people participate and estimate reliably. Here, all the assumptions are taken into consideration, thoroughly discussed, and agreed upon.

Advantages:

• This technique is particularly useful when dealing with estimating an effort for a task.
• The technique is completely anonymous and therefore everyone expresses their opinion.
• The technique is relatively simple.
• All the assumptions are equally discussed as well as documented.

Disadvantages:

• Management may not agree with the estimation every time.

2. Use-Case Point Method: Use-Case Points (UCP) method is an estimation technique used for measuring software with different use cases. This is a very simple technique as there is no additional analysis required.

Advantages:

• They are based on use cases and hence can be measured earlier in the Software project lifecycle.
• This technique is independent of skill, size, and experience.
• Estimations are close to actual estimations as they are implemented by experienced team members.

Disadvantages:

• The use cases of the UCP techniques are not uniformly structured.
• There is a high impact of Technical and environmental factors on the UCP method.
• More suited when the requirements are in the form of use cases and fail when the method is applied for assigning work within the Team.

3. Experience-Based Method: In this method, estimation is banked on the experience of similar projects executed by previous QA teams. This method assumes that the QA team has already tested a project of similar characteristics and uses whether estimating with this technique is feasible or not.

Advantages:

• High precision is achieved using this estimation technique.
• Faster Expert based estimation.

Disadvantages:

• This technique is not suitable for teams with less skilled resources.
• For unique projects, this technique is difficult to apply.