Run Tests in Parallel with PyTest

Testing is one of the key factors of the application and code development. It is necessary whenever a team or an individual aims for a stable and performance-efficient product in a production environment. It also acts as a confidence booster for the developers as well as makes code modifications and debugging easier. There are different testing frameworks available in the market for different coding languages. Pytest is one of the tools to test python code in an efficient manner. Enabling parallelization boosts the testing process to a great extent in Python. It lets a number of workers work in parallel and test different parts of the code simultaneously, enabling better utilization of resources.

Pytest and Parallelisation

Pytest is a unit testing tool/library which supports functional, integration as well as full E2E run testing in Python. This tool is free and easy to use with a big community of developers working around it to make it more stable. But, to run each test in synchronization creates a big havoc while working on large-scale projects and is also termed as a bad resource utilization. Hence, parallelization of testing acts as a big relief to solve this problem. The pytest-xdist library, which is a mere extension of PyTest, helps run the tests in parallel mode and hence, reduces the large execution time.

Steps to Run Tests with PyTest

Below are the steps by which we can use PyTest in Python:

Step 1: Installation

Use the following command to install PyTest on your local system.

pip3 install pytest

Step 2: Write Program Code ( my_math.py )

To use pytest, we require a Python program to be tested. Let’s create a basic python program with addition function for the same. Refer the following python code stored in my_math.py file.

Python3

# my_math.py
def add(a, b):
    return a + b

Step 3: Write Test File Code ( test.py )

Now we are ready to test this python function using pytest. Let’s create a new test file called test.py and import my_math.py file into it. We will write 5 different tests to test this python function.

Python3

from my_math import add

def test_add_positive_numbers():
    result = add(2, 3)
    assert result == 5

def test_add_negative_numbers():
    result = add(-2, -3)
    assert result == -5

def test_add_mixed_numbers_1():
    result = add(2, -3)
    assert result == -1

def test_add_mixed_numbers_2():
    result = add(-2, 3)
    assert result == 1

def test_add_zero():
    result = add(0, 0)
    assert result == 0

Step 4: Run Tests

With this, we are good to run the pytest for our python code. Use the following command to test code using pytest.

pytest test.py

Output

Refer the above screenshot to check the output of the above command. As can be seen in the output, everything has run successfully and the tests have passed.

Steps to Run Pytest Tests in Parallel

Step 1: Installation

Here, pytest-xdist does parallelism while pytest-parallel does parallelism and concurrency. To install the pytest-xdist library use the following command.

pip3 install pytest-xdist

Step 2: Write Program Code ( my_math.py )

To use pytest, we require a python program to be tested. Let’s create a basic python program with addition function for the same. Refer the following python code stored in my_math.py file.

Python3

def add(a, b):
    return a + b

Step 3: Write Test File ( test.py )

As can be seen in the previous output, it is very difficult to analyse time taken to run the tests on a small scale code. To make it easier, let’s add a sleep time of 5 seconds in each test to understand and analyse the total time taken to run 6 tests.

Python3

from my_math import add
import time

def test_add_positive_numbers():
    result = add(2, 3)
    time.sleep(5)
    assert result == 5

def test_add_negative_numbers():
    result = add(-2, -3)
    time.sleep(5)
    assert result == -5

def test_add_mixed_numbers_1():
    result = add(2, -3)
    time.sleep(5)
    assert result == -1

def test_add_mixed_numbers_2():
    result = add(-2, 3)
    time.sleep(5)
    assert result == 1

def test_add_zero():
    result = add(0, 0)
    time.sleep(5)
    assert result == 0

Step 4: Run Tests

Run the tests again using pytest command to find out the total time taken by library to run 5 tests. Refer the screenshot below, it takes around 30 seconds to fininsh the testing when run in a synchronised manner. Now, let’s use parallelisation in order to run the tests. Use the following command to tun tests in parallel.

pytest test.py -n <number of workers>

where <number of workers> is the number to be provided at run time to specify the number of threads which will run in paralell to execute testing.

Check the above output for the same code when run for 5 threads and 10 threads respectively. The time difference between the execution of tests when run parallely can be visibly seen through the output.