Formal Testing

Last Updated : 22 Oct, 2022

Formal testing is a type of software testing in which the testing of the software is done with proper planning and with proper documentation of its test cases.

What is Formal Testing?

In formal testing, the degree of thoroughness and formality of test cases depend upon the requirements of the project. Formal testing follows a systematic process called Software Testing Life Cycle (STLC).

There are multiple steps taken in formal testing:

Requirement Analysis.
Planning about tests.
Documentation of Test Cases.
Setting up the Test Environment.
Execution of Tests.
Closure of Tests.

There are two ways for testing software. One is the traditional manual testing way and the other one is the automated testing way. We can use traditional manual testing for both formal testing of the software and informal testing of the software. However, the automated testing method is only suitable for formal testing of software because of the defining set of rules.

Manual Testing: It is done with a person’s help, by clicking and interacting with the software using proper tools. The process is prone to error since it requires someone to set up an environment and perform the tests themselves. Since the testers are human it is prone to human error as the tester might make spelling mistakes or maybe skip the steps described in the test script.
Automated Testing: It is done with the help of machines that runs a test script that is already written for it in advance. These tests can differ in complexity. Automated testing is much more robust and reliable than manual testing but the quality of tests depends upon how the test scripts were written for the machine.

Formal Testing vs Informal Testing

Below are the differences between Formal Testing and Informal Testing:

Formal Testing: In Formal Testing, the tests on software are performed with proper planning and with proper documentation of its test cases. In this form of software testing, the test cases are developed using multiple methods such as domain coverage, equivalence partitioning, boundary testing, etc., and then those test cases along with a systematic approach known as STLC, are used to test the functionality of the software.

Informal Testing: In Informal Testing which is also known as Ad-hoc testing, the tests on software are performed informally and randomly without any proper planning or documentation after the formal testing is completed to find out any loophole in the system. It is a considerably cheaper mode of testing since it doesn’t require any proper planning or documentation and it also takes lesser time to complete.

Property	Formal Testing	Informal Testing
Definition	It is a mode of testing in which proper planning and documentation of the test cases are done before testing the software.	It is a mode of testing in which there is no prior planning nor there are any documented test cases of the software before testing the software.
Automation in the test process	It is possible to automate Formal testing because it follows a set of rules which is ideal for automation.	It is not possible to automate Informal Testing because this mode of testing does not follow any fixed set of rules.
Use Case	When an organization has enough time, resources, and proper documentation for software’s test case they choose this mode of testing.	When an organization does not have enough time, resources, or any kind of documentation regarding the software’s test cases they choose this mode of testing.
Cost	Formal testing methods are expensive as they include the cost of preparing the test cases, training the developers and testers, etc.	Informal testing methods are cheaper than formal testing as they do not include any documentation, training, or planning costs.
Example	An example of a formal testing method is the Software Testing Life Cycle.	An example of an informal testing method is Ad-hoc testing.

Where to use Formal Testing?

Proper Documentation: When an organization has proper documentation and numerous test cases they choose the mode of formal testing.
Enough Time: When an organization has enough time for testing the software then they choose the mode of formal testing.
Enough Resources: When an organization has enough resources for testing the software they choose the mode of formal testing.

It produces far better results than informal testing and there is little to no chance of any bugs getting left in the software after a formal test but if the organization does not have time, resources, or proper documentation for the software under development then they go for the informal mode of testing.

Formal Testing Process

Formal testing follows a systematic approach known as STLC so naturally, it contains all the steps involved in it.

Fig 1: Steps involved in Formal Testing

1. Requirement Analysis: In this phase, the testing team understands the requirements like what is to be tested. If anything is missing or not understandable then the testing team meets with the stakeholders to better understand the detailed knowledge of the requirement.

2. Planning Tests: The proper planning for software testing involves the formulation of a test objective, test strategy, and schedule. A plan helps developers of the software in many ways like analyzing the effort required to validate the quality of the software, fixing a schedule for appropriate testing of the software, etc. The planning of software testing involves multiple steps such as:

Product Analysis: To understand the software, for example, the GFG application, first of all, we need to know everything that is to know about the application itself after that we need to know about the users who use the application and what are their expectations from the application.

Fig 2: Product Analysis Cycle of GFG application

Defining Objectives: The objective of a test is to find as many problems present in the software as possible before it gets released to the public so that it can be bug-free and easy to use. The test objective can also be chosen on the basis of memory usage, the application’s functionality, its performance, etc.
Deciding Test Criteria: Test criteria are situations according to which the decision about the testing of the software can be made. It is of two types:
- Suspension: The suspension criteria tells that if the failure rate of the testing of a specific software project reaches a particular threshold then the testing of the software should be stopped temporarily until those errors are fixed and the test result should be marked as failed.
- Exit: The exit criteria tell that if the success rate of the testing of a specific software project reaches a particular threshold then the testing of the software should be stopped and the results of the testing should be marked as passed.
Strategizing the test: Strategizing involves choosing the scope of the tests and the efforts that will be required in them. It consists of four steps which are:
- Defining the scope of the test: Scope refers to the parts of the software that will be allowed to be tested during the testing period. It may be the front end of the application, the back end of the application, or maybe something totally different.
- Identifying the test type: The type of testing needs to be determined so that we can perform the correct test at the correct time. For example, for testing only the front or back end we will be doing unit testing for both of them and when we will be testing the entire application we will be doing integration and other tests.
- Risk assessment: Identifying the risks associated with the project is also an important task during planning. The developers do the feasibility study of the project to find the best approach for developing the software with the least amount of risk involved.
- Selecting the testers: Choosing the right testers for the software is also important. The skills belonging to the tester must qualify to support the results produced by him/her during the test.

3. Documentation of Test Cases: Documentation of a test case can be developed from design requirements, domain coverage, equivalence partitioning, boundary testing, etc.

Design requirement: It is the feature the developers are trying to create using the software. The test cases can be documented on how they meet the design requirements specified for the software under development.
Domain Coverage: It is a type of white box testing in which the software under development is provided with a minimum number of test inputs and its output is verified to look whether it accepts any values that are out of bound or out of range to be considered as a valid input to the software.
Equivalence Partitioning: It is a type of black box testing in which the input data is divided into equivalent valid and invalid parts from which test cases are derived.
Boundary Testing: It is a type of black box testing where boundary values of valid and invalid partitions are provided as input to the software to look for defects in the output as edge cases are more likely to produce errors than values inside partitions.

4. Setting Up the Test Environment: The test environment is defined as the place that may be hardware or software where the software under development will be tested. The environment simulates the appropriate network configuration required by the software to be tested to its full potential. It is an independent activity and can be started along with test case development. In this process, the testing team is not involved. either the developer or the customer creates the testing environment.

5. Execution of Tests: After the development of the test cases and setting up the test environment, the test execution step takes place. In this step, the testing team starts executing the test cases based on the prepared test cases in the earlier steps.

6. Closure of Tests: This is the last step of a formal testing process where the testing is analyzed.

Advantages of Formal Testing

Below are some of the advantages of formal testing:

Uncovers ambuiguity: It uncovers the ambiguity, and inconsistent and unfinished data in the software.
Gets better in every iteration: It provided better results every time the steps are repeated.
Less Complexity: With proper testing and documentation, the complexity is reduced.
Bug-Free: It results in a version of the software that is almost free from any bugs.

Disadvantages of Formal Testing

Below are some of the disadvantages of formal testing:

Time Consuming: Formal testing process takes a lot of time due to its long list of planning and documentation.
Expensive: It is also a very expensive task due to the cost of all the test case generation, documentation, and planning.
Difficult to implement: It is difficult to implement with non-technical staff who don’t have proper knowledge about the software.
Requires Training: The testers and developers working on the automation of testing need to be trained since developers having the knowledge of implementing formal modes of testing is very limited.