Tips to Improve the Performance of Python Application

Python…We all know the popularity of this language. Python is a powerful higher-order programming language. In today’s tech world, it’s almost everywhere. Whether you’re building a web application or working with machine learning, this language has become the first choice for developers. When it comes to optimizing developer productivity, Python comes first. You can create a program instantly and solve the business problems of your client. 

Writing a solution in Python doesn’t give you a guarantee that it is optimized, and it improves Python performance. However, while writing your code in Python, you can follow some strategies to help you with boosting the performance of your application. These strategies can make your application faster. 

Some tips or strategies will have a big impact on execution and others will have smaller, more subtle effects. Let’s discuss those tips in detail…

1. Use Built-In Functions

Built-in function in any language is always useful because you don’t need to write your code from scratch. The same goes for Python. Python comes with many useful libraries and built-in functions. These libraries are helpful in writing the features at several places in your development project. You can write high-quality, efficient code, but it is difficult to beat the underlying libraries. 

Python libraries are optimized and tested rigorously (like your code). These built-in functions are easy to use in your project. You won’t have redundant code in your project and the code will be optimized very well.

2. Write Your Own Generator

In Python use generator wherever it is possible. It allows you to return a single item at a time instead of returning the items all at once. Xrange() function is a generator in Python 2, similar to the range() function in Python 3.

If you’re using lists, you should write your own generator. Generators give you lazy evaluation and memory will be used efficiently. Generators are very useful if you’re reading numerous large files. You can process a single chunk without worrying about the size of the files.

Below is an example for your help…

import requests
import re

def get_pages(link):
  pages_to_visit = []
  pages_to_visit.append(link)
  pattern = re.compile('https?')
  while pages_to_visit:
    current_page = pages_to_visit.pop(0)
    page = requests.get(current_page)
    for url in re.findall('<a href="([^"]+)">', str(page.content)):
      if url[0] == '/':
        url = current_page + url[1:]
      if pattern.match(url):
        pages_to_visit.append(url)
    yield current_page
webpage = get_pages('http://www.example.com')
for result in webpage:
  print(result)

The above example returns a page at a time performing an action of some sort. In the above case, we are printing the link. Without a generator, you need to fetch and process the data at the same time or gather all the links before you start processing. In this case, your code will be cleaner, faster, and easier to test.

3. Use List Comprehensions

Just like in any other language, using loops is common in Python. You might have used list comprehensions in Python. List comprehension is a great way to execute your code faster. List comprehensions are concise and creating a new list is easier as it speeds up the process. Suppose, you want to find the square of all the odd numbers in a specific range. You can solve this problem using the loop as given below…

square_numbers = []
  for n in range(0,10):
    if n % 2 == 1:
      square_numbers.append(n**2)

You can solve the same problem, using the list comprehension in just one line…

square_numbers = [n**2 for n in range(1,10) if n%2 == 1]

You can see that the second approach is more optimized and readable. You will get the faster results after running this code. All these tips in small codebases have small ranges. This approach won’t make much difference but in other situations, it can make all the difference when you will try to save some time. Your program will run faster. 

4. Use xrange() Instead of range()

In Python 2, to iterate over loops we can use range() and xrange() functions. The first function stores all the numbers in the range in memory, and it got linearly large as range did. Other function xrange() return the generator object. If you loop with these object numbers will be available in memory only on demand. 

import sys
counter = range(1, 70000)
print(sys.getsizeof(counter))

The above code returns 560064. If you use xrange with the same range, it will return 40. If you’re writing your application in Python 2, Then swapping functions will create a big impact on memory usage. In Python 3 xrange() functionality is implemented by default. If there is no xrange() function, then the range() function will act like this. 

5. Use Sets and Unions

If you’re using too many loops in your code, then you will put unnecessary strain on your server. This will be the most efficient approach. You can get the overlapping values in two lists. You can do this using nested for loops, as given below…

a = [7,8,1,0,2]
b = [2,8,9,1,3]

overlaps = []
for x in a:
  for y in b:
    if x==y:
      overlaps.append(x)

print(overlaps)

The above code will print the list [8, 1, 2]. Here the number of comparisons will get very large, very quickly. Let’s see another approach…

a = [7,8,1,0,2]
b = [2,8,9,1,3]

overlaps = set(a) & set(b)

print(overlaps)

The above code will print the dictionary {8, 1, 2}. Here you will get a good speed and memory bump as a result.

6. Be Lazy With Module Importing

Experts in Python suggest importing all the modules at the start of your program. You can sort these alphabetically. This approach will help you in keeping the track of the dependencies your program has, but the disadvantage is that your imports load at startup. 

You can try a different approach? You can load the modules when you need them. This technique will help you in distributing the loading time for modules evenly, This will reduce peaks of memory usage.

7. Use “in” if Possible

It is recommended to use the “in” keyword to check the membership of a list. 

for username in member_list:
  print('{} is a member'.format(username))

Conclusion

All the above code will help you in running your code faster, and this will allow you to get a better Python performance from your application. As you will progress with Python, you will find many tips, and you on your own will learn to optimize your code in Python. You will be using lists, dictionaries, tuples, and many other things in Python and while building your logic or adding some features, slowly you will get to know that how things can be optimized.