Break a list into chunks of size N in Python

In this article, we will cover how we split a list into evenly sized chunks in Python.

Below are the methods that we will cover: 

• Using yield
• Using for loop in Python
• Using List comprehension
• Using Numpy
• Using itertool

Method 1: Break a list into chunks of size N in Python using yield keyword

The yield keyword enables a function to come back where it left off when it is called again. This is the critical difference from a regular function. A regular function cannot comes back where it left off. The yield keyword helps a function to remember its state. The yield enables a function to suspend and resume while it turns in a value at the time of the suspension of the execution. 

Python3

my_list = ['geeks', 'for', 'geeks', 'like',
           'geeky','nerdy', 'geek', 'love',
               'questions','words', 'life']

# Yield successive n-sized
# chunks from l.
def divide_chunks(l, n):

    # looping till length l
    for i in range(0, len(l), n):
        yield l[i:i + n]

# How many elements each
# list should have
n = 5

x = list(divide_chunks(my_list, n))
print (x)

Output:

[['geeks', 'for', 'geeks', 'like', 'geeky'], 
 ['nerdy', 'geek', 'love', 'questions', 'words'], 
 ['life']]

Time Complexity: O(n)
Auxiliary Space: O(1)

Method 2: Break a list into chunks of size N in Python using a loop

In this example, we are using a loop in python and list slicing that will help us to break a list into chunks.

Python3

my_list = [1, 2, 3, 4, 5,
           6, 7, 8, 9]
start = 0
end = len(my_list)
step = 3
for i in range(start, end, step):
    x = i
    print(my_list[x:x+step])

Output:

[1, 2, 3]
[4, 5, 6]
[7, 8, 9]

Time Complexity: O(n)
Auxiliary Space: O(n), where n is length of list

Method 3: Break a list into chunks of size N in Python using List comprehension

It is an elegant way to break a list into one line of code to split a list into multiple lists in Python.

Python3

my_list = [1, 2, 3, 4, 5,
              6, 7, 8, 9]

# How many elements each
# list should have
n = 4

# using list comprehension
final = [my_list[i * n:(i + 1) * n] for i in range((len(my_list) + n - 1) // n )]
print (final)

Output:

[[1, 2, 3, 4], [5, 6, 7, 8], [9]]
Auxiliary Space: O(1)

Alternate Implementation : 

Python3

l = [1, 2, 3, 4, 5, 6, 7, 8, 9]

# How many elements each
# list should have
n = 4

# using list comprehension
x = [l[i:i + n] for i in range(0, len(l), n)]
print(x)

Output:

[[1, 2, 3, 4], [5, 6, 7, 8], [9]]

Method 4: Break a list into chunks of size N in Python using Numpy

Here, we are using a Numpy.array_split, which splits the array into n chunks of equal size.

Python3

import numpy as np

arr = range(30)
np.array_split(arr, 6)

Output:

[array([0, 1, 2, 3, 4]),
 array([5, 6, 7, 8, 9]),
 array([10, 11, 12, 13, 14]),
 array([15, 16, 17, 18, 19]),
 array([20, 21, 22, 23, 24]),
 array([25, 26, 27, 28, 29])]

Method 5: Break a list into chunks of size N in Python using itertool

In this example, we will use itertool to slice each array in equal size. we are passing a parameter in the range of 30 and split size of 5.

Python3

from itertools import islice


def chunk(arr_range, arr_size):
    arr_range = iter(arr_range)
    return iter(lambda: tuple(islice(arr_range, arr_size)), ())


list(chunk(range(30), 5))

Output:

[(0, 1, 2, 3, 4),
 (5, 6, 7, 8, 9),
 (10, 11, 12, 13, 14),
 (15, 16, 17, 18, 19),
 (20, 21, 22, 23, 24),
 (25, 26, 27, 28, 29)]

Method 6: Collections

One approach to splitting a list into chunks of size N without using a loop is to use the collections module. The collections module has a deque class that allows you to easily split a list into chunks of a specific size.

Here’s an example of how you can use the deque class to split a list into chunks of size N:

Python3

from collections import deque

def split_list(input_list, chunk_size):
  # Create a deque object from the input list
  deque_obj = deque(input_list)
  # While the deque object is not empty
  while deque_obj:
      # Pop chunk_size elements from the left side of the deque object
      # and append them to the chunk list
      chunk = []
      for _ in range(chunk_size):
        if deque_obj:
          chunk.append(deque_obj.popleft())

      # Yield the chunk
      yield chunk
input_list = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
chunk_size = 3
chunks = list(split_list(input_list, chunk_size))
print(chunks) # [[1, 2, 3], [4, 5, 6], [7, 8, 9], [10]]

Output

[[1, 2, 3], [4, 5, 6], [7, 8, 9], [10]]

The deque class allows you to easily remove elements from the left or right side of the list, making it easy to split the list into chunks of a specific size. The code uses a while loop and a generator function to iterate over the list and yield the chunks one at a time. The loop breaks when the deque is empty, which indicates that all elements have been processed.

This approach has a time complexity of O(n) and a space complexity of O(n), where n is the size of the input list.

Method 7: Partial assignment

Here’s an example of how you can easily process list by chunks of size N:

Python3

my_list = list(range(10))
chunk_size = 3
while my_list:
    chunk, my_list = my_list[:chunk_size], my_list[chunk_size:]
    print(chunk)

Output

[0, 1, 2]
[3, 4, 5]
[6, 7, 8]
[9]