How to Parallelize a While loop in Python?

Parallelizing a while loop in Python involves distributing the iterations of a loop across multiple processing units such as the CPU cores or computing nodes to execute them concurrently. This can significantly reduce the overall execution time of the loop, especially for tasks that are CPU-bound or require heavy computation. In this article, we’ll explore various approaches to parallelizing a while loop in Python using different libraries and techniques.

How to Parallelize a While loop in Python?

Below, are the methods to explain how to Parallelise a while loop in Python:

Parallelize a While loop Using concurrent.futures

In this example, the below code uses ThreadPoolExecutor to parallelize a loop in Python. The function GFG represents the task to be performed in each iteration. The Geek function defines the loop’s range and submits tasks to the executor for parallel execution. Each task corresponds to an iteration of the loop, processing a specific range defined by the start and end parameters.

Python3

from concurrent.futures import ThreadPoolExecutor

def GFG(start, end):

    print(f"Processing range ({start}, {end})")

def Geek():

    start = 0
    end = 5
    with ThreadPoolExecutor(max_workers=4) as executor:

        for i in range(start, end):
            executor.submit(GFG, i, i+1)

if __name__ == "__main__":
    Geek()

Output

Processing range (0, 1)
Processing range (1, 2)
Processing range (2, 3)
Processing range (3, 4)
Processing range (4, 5)

Parallelize a While loop Using Multiprocessing

In this example, below code parallelizes a While loop in Python using the multiprocessing module. It defines a function parallel_while_loop() which creates separate processes for each iteration of the loop using the Process class. Each process executes the GFG() function with iteration parameters.

Python3

from multiprocessing import Process


def GFG(start, end):

    # Perform computation or task within the loop
    print(f"Processing range ({start}, {end})")


def parallel_while_loop():

    # Define the range of iterations for the loop
    start = 0
    end = 5

    # Create a separate process for the each iteration
    processes = []
    for i in range(start, end):
        process = Process(target=GFG, args=(i, i+1))
        processes.append(process)
        process.start()

    # Wait for all processes to finish
    for process in processes:
        process.join()


if __name__ == "__main__":
    parallel_while_loop()

Output

Processing range (0, 1)
Processing range (1, 2)
Processing range (2, 3)
Processing range (4, 5)
Processing range (3, 4)