Mutation Testing using Mutpy Module in Python

Prerequisite: Mutation Testing

Mutpy is a Mutation testing tool in Python that generated mutants and computes a mutation score. It supports standard unittest module, generates YAML/HTML reports and has colorful output. It applies mutation on AST level.

Installation:

This module does not come built-in with Python. To install it type the below command in the terminal.

pip install mutpy

Here, we will use the Mutpy library to execute test cases written for a simple program.

Now, we perform Mutation Testing for a program that checks if a number is prime or not



Code: This code is save as isPrime.py

Python3

filter_none

edit
close

play_arrow

link
brightness_4
code

# define a function 
def isPrime(num):
  
    if num > 1:
        
       # check for factors
       for i in range(2,num):
           if (num % i) == 0:
               return False
  
       else:
           return True
             
    # if input number is less than
    # or equal to 1, it is not prime
    else:
       return False;

chevron_right


Now, we need to write test cases for the above program using Pytest or Unittest library. Test cases should be written with an approach to kill all the mutants, i.e. test cases should be effective enough to give a good mutation score. The test cases are written using the Unittest library in the file shown below. 

Test cases are written using self.assertEqual() that is a test assertion generally used to determine if a test case has passed or failed. We have written three test functions below to check the three types of input: 

  1. Input is prime
  2. Input is non-prime
  3. Input is invalid

Note: The function name and the test file name should always start with the word ‘test’.

Code: This code is save as test_isPrime.py.

Python3

filter_none

edit
close

play_arrow

link
brightness_4
code

# import required libraries
from unittest import TestCase
from isPrime import isPrime
  
# define a class
class CalculatorTest(TestCase):
      
    # test case for checking non prime nums
    def test_nonprime(self):
        self.assertEqual(isPrime(12),False)
      
    # test case to check prime nums
    def test_prime(self):
        self.assertEqual(isPrime(19),True)
  
    # test case to check invalid input
    def test_invalid(self):
        self.assertEqual(isPrime(-1),False)

chevron_right


To execute these test cases, we need to create two separate files isPrime.py and test_isPrime.py in a single folder and run the following command in the command prompt: 

mut.py --target isPrime --unit-test test_isPrime -m --runner pytest

In the above command, we have to specify three things: 

  • Target: the target file on which test cases will be run, which in our case is isPrime.py
  • Unit-test: the file which has the unit tests that have to be executed, i.e. test_isPrime.py in our case.
  • Runner: pytest or unittest

The output will be a set of mutants along with details like mutation score, the number of mutations killed, survived, etc. 

[*] Start mutation process:
   - targets: isPrime
   - tests: test_isPrime
[*] 3 tests passed:
   - test_isPrime [3.07469 s]
[*] Start mutants generation and execution:
   - [#   1] AOR isPrime:
--------------------------------------------------------------------------------
   3:
   4:     if num > 1:
   5:
   6:         for i in range(2, num):
-  7:             if num % i == 0:
+  7:             if num * i == 0:
   8:                 return False
   9:         else:
  10:
  11:             return True
--------------------------------------------------------------------------------
[1.45151 s] killed by testing program mutpy/test_isPrime.py::CalculatorTest::test_nonprime
   - [#   2] COI isPrime:
--------------------------------------------------------------------------------
   1:
   2: def isPrime(num):
   3:
-  4:     if num > 1:
+  4:     if not (num > 1):
   5:
   6:         for i in range(2, num):
   7:             if num % i == 0:
   8:                 return False
--------------------------------------------------------------------------------
[1.25723 s] killed by testing program mutpy/test_isPrime.py::CalculatorTest::test_invalid
   - [#   3] COI isPrime:
--------------------------------------------------------------------------------
   3:
   4:     if num > 1:
   5:
   6:         for i in range(2, num):
-  7:             if num % i == 0:
+  7:             if not (num % i == 0):
   8:                 return False
   9:         else:
  10:
  11:             return True
--------------------------------------------------------------------------------
[1.28817 s] killed by testing program mutpy/test_isPrime.py::CalculatorTest::test_prime
   - [#   4] CRP isPrime:
--------------------------------------------------------------------------------
   1:
   2: def isPrime(num):
   3:
-  4:     if num > 1:
+  4:     if num > 2:
   5:
   6:         for i in range(2, num):
   7:             if num % i == 0:
   8:                 return False
--------------------------------------------------------------------------------
[1.23510 s] survived
   - [#   5] CRP isPrime:
--------------------------------------------------------------------------------
   2: def isPrime(num):
   3:
   4:     if num > 1:
   5:
-  6:         for i in range(2, num):
+  6:         for i in range(3, num):
   7:             if num % i == 0:
   8:                 return False
   9:         else:
  10:
--------------------------------------------------------------------------------
[1.20360 s] survived
   - [#   6] CRP isPrime:
--------------------------------------------------------------------------------
   3:
   4:     if num > 1:
   5:
   6:         for i in range(2, num):
-  7:             if num % i == 0:
+  7:             if num % i == 1:
   8:                 return False
   9:         else:
  10:
  11:             return True
--------------------------------------------------------------------------------
[1.23499 s] killed by testing program mutpy/test_isPrime.py::CalculatorTest::test_prime
   - [#   7] ROR isPrime:
--------------------------------------------------------------------------------
   1:
   2: def isPrime(num):
   3:
-  4:     if num > 1:
+  4:     if num < 1:
   5:
   6:         for i in range(2, num):
   7:             if num % i == 0:
   8:                 return False
--------------------------------------------------------------------------------
[1.24164 s] killed by testing program mutpy/test_isPrime.py::CalculatorTest::test_invalid
   - [#   8] ROR isPrime:
--------------------------------------------------------------------------------
   1:
   2: def isPrime(num):
   3:
-  4:     if num > 1:
+  4:     if num >= 1:
   5:
   6:         for i in range(2, num):
   7:             if num % i == 0:
   8:                 return False
--------------------------------------------------------------------------------
[1.21934 s] survived
   - [#   9] ROR isPrime:
--------------------------------------------------------------------------------
   3:
   4:     if num > 1:
   5:
   6:         for i in range(2, num):
-  7:             if num % i == 0:
+  7:             if num % i != 0:
   8:                 return False
   9:         else:
  10:
  11:             return True
--------------------------------------------------------------------------------
[1.32597 s] killed by testing program mutpy/test_isPrime.py::CalculatorTest::test_prime
[*] Mutation score [14.91747 s]: 66.7%
   - all: 9
   - killed: 6 (66.7%)
   - survived: 3 (33.3%)
   - incompetent: 0 (0.0%)
   - timeout: 0 (

We can see from the output above, 6 mutants have been killed and only 3 mutant was able to survive. Also, a mutation score of 66.7% was achieved. We can further improve this mutation score by analyzing the mutants that survived the test cases and writing new or modifying test cases to kill the mutants that survived. 

Attention geek! Strengthen your foundations with the Python Programming Foundation Course and learn the basics.

To begin with, your interview preparations Enhance your Data Structures concepts with the Python DS Course.




My Personal Notes arrow_drop_up

Check out this Author's contributed articles.

If you like GeeksforGeeks and would like to contribute, you can also write an article using contribute.geeksforgeeks.org or mail your article to contribute@geeksforgeeks.org. See your article appearing on the GeeksforGeeks main page and help other Geeks.

Please Improve this article if you find anything incorrect by clicking on the "Improve Article" button below.


Article Tags :

Be the First to upvote.


Please write to us at contribute@geeksforgeeks.org to report any issue with the above content.