Software Testing – Test Plan, Estimates and Strategy
Working as a Test lead or a test manager means being involved in planning, monitoring, and control of the testing activities and tasks as a fundamental activity. During the test planning phase, Test lead needs to collaborate with other stakeholders like the Development team, Client, business to devise the test objectives, test strategies, and test plan. Sometimes they have to work upon creating Test policies as well if one is not already in place.
Test manager also needs to estimate the testing which is required to be done and derive the resources which might be required to achieve the testing. The resources can be human resources or tools. They also need to work upon and identify when automation is necessary. If the automation deems to be necessary, they need to work on identifying the tool, identify and train resources. During the execution phase of testing, they need to ensure the environment availability before and during the test cycle.
All the above-mentioned activities are figured and documented in various artifacts which are called by names of Test Plan, Test Strategy, Test Estimates, and Test Policies. Let us go through each one of them and understand the difference and how they are related to each other.
1. Test Policy :
- Test Policy basically gives you rules of testing. It is a document that is defined at an organizational level and gives you insights on what the organization’s stand on testing activities is and how the organization measures test “success”.
- It is not defined for each project undertaken by the organization separately.
- Hence, we can say the audience is an entire organization. It is defined by the senior leadership of the organization and “has to be” followed by each testing project an organization undertakes. It covers the “Why” of testing.
- It is therefore intended to provide the foundation on which all the subsequent test strategies and test plans be built.
- It is a short and crisp document – usually one or two pages. It lies at the top of the test documentation pyramid.
- More often than not, the Test policy document is missing and hence creates real problems for organizations.
For example- In defect detection, the test policy might mention defect detection percentage as an effectiveness metric and the cost of defects detected in testing vs defects detected after release as efficiency metrics.
2. Test Strategy :
- It is a document that is prepared at an organizational level. It is also a high-level document.
- It is independent of any specific project and covers the “how” of testing for an organization. So, The test strategy is the organization’s method of testing.
- It can cover the options that the organization can use to cover the risks during testing.
- It can provide the detail on the various testing levels or phases that an organization typically uses on any given project.
- A test strategy can describe both Short Term and long-term needs. As the needs change, Strategy should address the reasons and how these changes can affect testing.
- Also, Strategy documents can be tailored based on project and application. For example- The strategy for a safety-critical application that requires adhering to compliance will differ from the strategy for an e-commerce application. Test plan lies
- It can cover test automation approach, Regression, and retesting approach, etc.
Examples of topics covered in test strategy are – It can describe how unit testing, integration testing, System testing, or user acceptance testing levels are managed. It can also cover general entry and exit criteria for each test level (say – 0 integration testing open defects is an entry criteria for entering the system test phase).
3. Test Plan :
It is a document that is prepared at a project level. It is a project plan for testing tasks at hand. Documenting a test plan helps to communicate with other teams within the project, the Manager, and other stakeholders.
Writing a test plan needs an organized approach and a good test plan should be short and crisp. Test plan covers –
- Test Objectives
- In-Scope and out of scope items
- Approach (extent of testing, automation/regression/performance test)
- Test Schedule
- Entry and Exit Criteria
- Pass/Fail criteria
- Test Deliverables (Reports, test cases, etc)
- Environmental needs
- Staffing needs
Please note that It does not cover any details of test cases or scripts. Also, note that for bigger projects, we might have a Master Test plan and granular test plans corresponding to test levels like performance test plan and integration test plan, etc.
4. Test Estimates :
Estimates create approximate cost and schedule targets for the testing tasks. A good estimate should be based on the knowledge of experienced practitioners and supported by the people who will actually do the work on the ground. It is based on the “most” likely cost, effort, and duration of each task.
There are multiple factors affecting estimations and these typically fall into four categories :
- Process factors –
These arise from processes by which work is accomplished. Example – Chosen SDLC, Proper execution of prior testing phases.
- Material Factors –
They arise from the nature of the project, tools at hand, and the resources available. Example- Reusable test system and documentation from similar previous projects.
- People factor –
These arise from people on the team. Example Skilled vs new team.
- Delaying factors –
These generally contribute to delays of a project like the complexity of the application, many sub-teams, multiple stakeholders, custom hardware, etc.
There are multiple test estimation techniques that can be adopted based on the project situation –
- Experience-based estimation
- Work Break down structure
- Test Point Analysis
- Taking Industry Averages
These techniques can be applied in Silo as well as together to achieve a robust estimate to the testing project.