When a test engineer prepares a test case, he or she may skip some scenarios like entering incorrect data, and writing incorrect navigation steps, all of which might have an impact on the overall test execution process. To avoid this, one round of evaluation and approval will be conducted before beginning the test. If some test cases are missed out and the review procedure is not done then the accuracy of the test case document will be less. Only after the test case has been written must all cases be sent for review to another test engineer, known as a reviewer, for review. In software testing, the examination of test cases is a critical step. Every functionality listed in the Software Requirement Specification is addressed by the test case. The test case should be effective as well as adhere to the test case writing guidelines. Each test case should be checked to ensure the success and thoroughness of the test. Here, we will discuss the following points:
- Why Review Test Cases?
- What is Test Case Repository?
- Benefits of Test Case Repository.
- Test Case Review Process.
- Techniques of Test Case Review.
- Tips While Reviewing Test Cases.
- Factors to Consider During Test Case Review.
- Common Mistakes During Test Case Review.
- Classifying Defects in Review of the Test Cases.
Why Review Test Cases?
Peer review should be done on any work product that is deemed a deliverable. Test cases, which are important deliverables for the testing team, are included in this category. It is critical to write effective test cases that successfully uncover as many faults as possible during the testing process. As a result, a check is required to determine whether:
- Test cases are developed with the intention of detecting faults, and the requirement is correctly understood.
- Areas of potential impact are identified and put to the test.
- The test data is accurate and covers every possible domain class. There are scenarios that are both favorable and negative.
- The expected behavior is accurately documented.
- The test coverage is sufficient.
What is Test Case Repository?
Keeping test cases organized pays off handsomely, particularly in medium to large projects. Furthermore, testing is a procedure that can be repeated. Everyone benefits from reusing test cases since it saves time. Large elements of projects can be repeated for testing. Maintaining a test case repository allows one to reuse past test resources as needed, which helps to save time. The good news is that keeping a well-organized test case repository isn’t difficult at all. The quantity and variety of test cases that constitute the basis of testing cycles are often linked to the success of a software testing team. The assimilation of test cases may take a significant amount of time and effort, with the main focus being on creating a complete test case repository for each application. Test cases in the repository cover all essential permutations and combinations in workflow execution and transaction, ensuring that all system and user interactions are covered.
- A test case repository is a centralized storage area for all baseline test cases (authored, reviewed, and authorized).
- When the client provides the requirements, the developer begins constructing the modules, and the test engineer begins writing the test cases.
- The authorized test cases are stored in a test case repository.
- If a test engineer wishes to test the application, he or she must use the test case repository to get the test case.
- We can remove test cases from the test case repository if we don’t need them.
- We have a separate test case repository for each version.
- Without the authorization of the test lead, test cases cannot be modified or changed once they have been baselined or saved in the test case repository.
- If there is a crash that affects the software, the testing team always has a complete backup of the test case repository.
Furthermore, because a test case repository increases over time, testers should keep it up to date with each new version of the business application or software product. If this is not done, it will lose synchronization with the software’s actual function and behavior over time. As a result, subsequent QA cycles’ findings will suffer as a result.
Benefits of Test Case Repository
The following are some of the benefits of a test case repository:
- The repository can be updated if something new comes up.
- Saves time.
- Test case writing skills of testers are evaluated on a performance basis.
- Coverage.
- Assists with reporting.
- Traceability.
- Charts can be created to display the status of test cases (pass, fail, not tested).
- Can assist in the development of new similar items.
- Less data is passed from one tester to the next.
Test Case Review Process
The following are the list of activities involved in the review process:
1. Planning: This is the first phase and begins with the author requesting a moderator for the review process. The moderator is responsible for scheduling the date, time, place, and invitation of the review. The entry check is done to make sure that the document is ready for review and does not have a large number of defects. Once the document clears the entry check the author and moderator decide which part of the document to be reviewed.
2. Kick-Off: This is an optional step in the review process. The goal is to give a short introduction on the objectives of the review and documents to everyone in the meeting.
3. Preparation: The reviewers review the document using the related documents, procedures, rules, and checklists provided. Each participant while reviewing identifies the defects, questions, and comments according to their understanding of the document.
4. Review Meeting: The review meeting consists of three phases-
- Logging phase: The issues and defects identified during the preparation phase are logged page by page. The logging is done by an author or scribe, where a scribe is a person to do the logging. Every defect and its severity should be logged.
- Discussion phase: If any defects need discussion then they will be logged and handled in this phase. The outcome of the discussion is documented for future purposes.
- Decision phase: A decision on the document under review has to be made by the participants.
5. Rework: If the number of defects found per page exceeds a certain level then the document has to be reworked.
6. Follow-Up: The moderator checks to make sure that the author has taken action on all known defects.
Test Case Review Process
Techniques for Test Case Review
There are three techniques to conduct a test case review:
- Self-review: This is carried out by the tester who wrote the test cases. By looking at SRS/FRD, he can see if all of the requirements are met or not.
- Peer review: This is done by another tester who isn’t familiar with the system under test but hasn’t developed the test cases. Maker and Checker are two other names for the same thing.
- Supervisory review: This is done by a team lead or manager who is higher in rank than the tester who wrote the test cases and has an extensive understanding of the requirements and system under test.
Tips While Reviewing Test Cases
Below are some of the tips to be kept in mind while reviewing the test cases:
- In the review process, it’s best to stick to version numbers. For example, if reviewing a test case plan for the first time, make it v.1. Once the tester has completed all of the changes, re-review it and make it v.1.1. One will be able to tell which one is the most recent this way, and there will be a complete record of the plan’s changes.
- It is always preferable to meet with the tester face to face to ensure that he fully comprehends all of the review input.
- If at all possible, run test cases on the SUT (System Under Test) to gain a better understanding of the outcomes and actions involved in their execution.
- It is preferable to have a copy of SRS/FRD with you while reading the test case for reference.
- If you are unsure about a test case or expected outcome, consult with the client or your supervisor before making a decision.
Factors to Consider During Test Case Review
During the review, the reviewer looks for the following in the test cases:
1. Template: The reviewer determines if the template meets the product’s requirements.
2. Header: The following aspects will be checked in the header:
- Whether or not all of the qualities are captured is debatable.
- Whether or not all of the traits are relevant is debatable.
- Whether all of the traits are filled or not is up to you.
3. Body: Look at the following components in the test case’s body:
- The test case should be written in such a way that the execution procedure takes as little time as possible.
- Whether or not all feasible circumstances are covered is debatable.
- Look for a flow that has the maximum amount of test coverage.
- Whether or not to use the test case design technique.
- The test case should be easy to comprehend.
| TestCaseName | Step No | Reviewer | Author Comments | Comments | |
|---|---|---|---|---|---|
| Comments | Severity | ||||
|
ST100 CNNB-ET001 |
7 | Module Not Linked | Critical | Precondition is Required | |
|
ST200 SNNC-XD007 |
45 | Invalid Parameter | Major | Fixed | ——– |
|
NJ120 BKKL-PP330 |
18 | Unused Variable | Minor | Fixed | ——— |
| 07 | Need more input value | Major | Not Fixed | ——– | |
4. Text Execution Report:
- It is the last document produced by a test lead after all of the testings has been finished.
- The test execution report defined the application’s stability and included data such as the number of cases written, ran, passed, and failed, as well as their percentages.
- The test execution report is a final summary report that defines the application’s quality and aids in determining whether or not the program may be handed over to the customer.
- Every module has its own spreadsheet to track its progress.
| Module Name | Total Test Cases | Total Test Cases Executed | Total Test Case Passed | Total Test Case Failed | Pass% | Fail% |
|---|---|---|---|---|---|---|
|
Marketing |
110 |
110 |
100 |
10 |
90% |
10% |
|
Payment |
200 |
200 |
150 |
50 |
75% |
25% |
|
Production |
70 |
70 |
70 |
0 |
100% |
0% |
|
Sales |
100 |
90 |
45 |
45 |
50% |
50% |
|
Loans |
120 |
120 |
100 |
20 |
80% |
20% |
|
Total = |
600 |
590 |
465 |
125 |
65% |
35% |
This report was written by the test lead, and the test engineer submitted the particular features that he or she had tested and implemented.
This report is sent to the following addresses by the test lead:
- Development Team.
- Management.
- Test manager.
- Customer.
Where the list of failed test cases is required by a development team. There is a list of test case names, related status, and comments, as shown in the table below. The data from the Sales test case is shown in the table below-
| Step Number | Test Case Name | Test Case Status | Comments |
|---|---|---|---|
| 1 |
ST100 CNNB-ET001 |
Pass | – |
| 2 |
ST200 SNNC-XD007 |
Pass | – |
| 3 |
ST200 SNNC-XD007 |
Failed | Bug |
| . | |||
| . | |||
| . | |||
| . | |||
| 98 | |||
| 99 | |||
| 100 |
Common Mistakes During Test Case Review
Below are some common mistakes checked during the test case review process:
- Spelling errors: Spelling errors can sometimes cause a lot of confusion or make a statement difficult to grasp.
- Replication of Test Cases: It relates to the reduction of redundant test cases. It’s possible that two or more test cases are testing the same item and can be combined into one, saving time and space.
- Standard/Guidelines: It’s critical to examine whether all of the standards and guidelines are being followed correctly during the review process.
- Redundancy: When a test case is rendered obsolete owing to a change in requirements or certain adjustments, it is referred to be redundancy. These types of test scenarios must be eliminated.
- The manner used: Test cases should be written in a basic, easy-to-understand language.
- Grammar: If the grammar is incorrect, the test case can be misinterpreted, leading to incorrect findings.
- Format of Template: When a suitable template is followed, it is simple to add/modify test cases in the future, and the test case plan appears orderly.
Classifying Defects in Review of the Test Cases
When these checklists are utilized consistently and problems are discovered, it is recommended that the defects be classified into one of the following categories:
- Incomplete test cases.
- Missing negative test cases.
- No test data.
- Inappropriate/Incorrect test data.
- Incorrect Expected behavior.
- Grammatical problems.
- Typos.
- Inconsistent tense/voice.
- Incomplete results/number of test runs.
- Defect information was not recorded in the test case.
Defects could sneak into production if test cases aren’t thoroughly reviewed. As a result, production issues could be reported, thereby impacting the Software’s quality. Resolving problems at this time would be much more expensive than fixing them if they had been discovered during the testing phase.