Python | Making program run faster

As we know, Python programming language is a bit slow and the target is to speed it up without the assistance of more extreme solutions, such as C extensions or a just-in-time (JIT) compiler.

While the first rule of optimization might be to “not do it”, the second rule is almost certainly “don’t optimize the unimportant.” To that end, if the program is running slow, one might start by profiling the code. More often than not, one finds that the program spends its time in a few hotspots, such as inner data processing loops. Once those locations are identified, the no-nonsense techniques can be used to make the program run faster.

A lot of programmers start using Python as a language for writing simple scripts. When writing scripts, it is easy to fall into a practice of simply writing code with very little structure.

Code #1 : Taking this code into consideration.

filter_none

edit
close

play_arrow

link
brightness_4
code

# abc.py
import sys
import csv
  
with open(sys.argv[1]) as f:
    for row in csv.reader(f):
        # Some kind of processing

chevron_right


A little-known fact is that code defined in the global scope like this runs slower than code defined in a function. The speed difference has to do with the implementation of local versus global variables (operations involving locals are faster). So, simply put the scripting statements in a function to make the program run faster.
 
Code #2 :

filter_none

edit
close

play_arrow

link
brightness_4
code

# abc.py
import sys
import csv
  
def main(filename):
    with open(filename) as f:
        for row in csv.reader(f):
        # Some kind of processing
  
main(sys.argv[1])

chevron_right


The speed difference depends heavily on the processing being performed, but the speedups of 15-30% are not uncommon.

Selectively eliminate attribute access –

Every use of the dot (.) operator to access attributes comes with a cost. Under the covers, this triggers special methods, such as __getattribute__() and __getattr__(), which often lead to dictionary lookups.

One can often avoid attribute lookups by using the from module import name form of import as well as making selected use of bound methods as shown in the code fragment given below –

Code #3 :

filter_none

edit
close

play_arrow

link
brightness_4
code

import math
  
def compute_roots(nums):
    result = []
    for n in nums:
        result.append(math.sqrt(n))
    return result
  
# Test
nums = range(1000000)
for n in range(100):
    r = compute_roots(nums)

chevron_right


Output :

This program runs in about 40 seconds when running on the machine.

Code #4 : Change the compute_roots() function

filter_none

edit
close

play_arrow

link
brightness_4
code

from math import sqrt
  
def compute_roots(nums):
    result = []
    result_append = result.append
    for n in nums:
        result_append(sqrt(n))
    return result

chevron_right


Output :

This program runs in about 29 seconds when running on the machine.

The only difference between the two versions of code is the elimination of attribute access. Instead of using math.sqrt(), the code uses sqrt(). The result.append() method is additionally placed into a local variable re
sult_append and reused in the inner loop.
However, it must be emphasized that these changes only make sense in frequently executed code, such as loops. So, this optimization really only makes sense in carefully selected places.

Understand locality of variables –

As previously noted, local variables are faster than global variables. For frequently accessed names, speedups can be obtained by making those names as local as possible.

Code #5 : Modified version of the compute_roots() function

filter_none

edit
close

play_arrow

link
brightness_4
code

import math
  
def compute_roots(nums):
    sqrt = math.sqrt
    result = []
    result_append = result.append
    for n in nums:
        result_append(sqrt(n))
    return result

chevron_right


In this version, sqrt has been lifted from the math module and placed into a local variable. This code will run about 25 seconds (an improvement over the previous version, which took 29 seconds). That additional speedup is due to a local lookup of sqrt being a bit faster than a global lookup of sqrt.
Locality arguments also apply when working in classes. In general, looking up a value such as self.name will be considerably slower than accessing a local variable. In inner loops, it might pay to lift commonly accessed attributes into a local variable as shown in the code given below.

Code #6 :

filter_none

edit
close

play_arrow

link
brightness_4
code

# Slower
class SomeClass:
    ...
    def method(self):
        for x in s:
            op(self.value)
# Faster
class SomeClass:
    ...
    def method(self):
        value = self.value
        for x in s:
            op(value)

chevron_right




My Personal Notes arrow_drop_up

Check out this Author's contributed articles.

If you like GeeksforGeeks and would like to contribute, you can also write an article using contribute.geeksforgeeks.org or mail your article to contribute@geeksforgeeks.org. See your article appearing on the GeeksforGeeks main page and help other Geeks.

Please Improve this article if you find anything incorrect by clicking on the "Improve Article" button below.




Article Tags :

Be the First to upvote.


Please write to us at contribute@geeksforgeeks.org to report any issue with the above content.