Memory profiling in Python using memory_profiler

If you use Python a lot then you probably know that many people claim that Python takes up more time to execute. Well, you probably have seen approaches like the total time spent to execute a part of code or something like that but sometimes you need something more than that. What about RAM usage? Nobody really talks about that but it’s equally essential. This might look really insignificant but it’s actually pretty important while writing code for production. 

We will be using memory-profiler from PyPi. We will also be needing requests to test the functionality. To do so, simply type the following in your terminal

pip3 install memory-profiler requests

Note: If you are working on windows or using a virtual env, then it will be pip instead of pip3

Now that everything is set up, rest is pretty easy and interesting obviously.  Create a new file with the name word_extractor.py and add the code to it. Below is the implementation of the code. Everything is pretty well documented as in-line comments.

Python3



filter_none

edit
close

play_arrow

link
brightness_4
code

# imports
from memory_profiler import profile
import requests
  
  
class BaseExtractor:
    # decorator which specifies which
    # function to monitor
    @profile
    def parse_list(self, array):
        
        # create a file object
        f = open('words.txt', 'w')
        for word in array:
              
            # writing words to file
            f.writelines(word)
      
    # decorator which specifies which 
    # function to monitor
    @profile
    def parse_url(self, url):
        
        # fetches the response
        response = requests.get(url).text
        with open('url.txt', 'w') as f:
              
            # writing response to file
            f.writelines(response)

chevron_right


Notice the @profile this is a decorator. Any function which is decorated by this decorator, that function will be tracked. Now, our main code is ready. Let’s write the driver code which will call this class functions. Now, create another file called run.py and insert the following code in it.

Python3

filter_none

edit
close

play_arrow

link
brightness_4
code

from word_extractor import BaseExtractor
  
  
if __name__ == "__main__":
      
    # url for word list (huge)
      
    # word list in array
    array = ['one', 'two', 'three', 'four', 'five']
      
    # initializing BaseExtractor object
    extractor = BaseExtractor()
      
    # calling parse_url function
    extractor.parse_url(url)
      
    # calling pasrse_list function
    extractor.parse_list(array)

chevron_right


So, basically now we are done. You will notice that parse_url() will consume more memory than parse_list() which is obvious because parse_url calls a URL and writes the response content to a text file. If you open the link, then you will find that the word list is huge. So, now to test your code, simply run the run.py  file. You can do so by typing 

python3 run.py

Note: If you are working on windows or using a virtual env, then it will be python instead of python3

If everything ran successfully, then you should see something like this

Memory Profiler Stats

An important thing to remember is that memory-profiler itself consumes a significant amount of memory. Use this only in development but avoid it in production.

Attention geek! Strengthen your foundations with the Python Programming Foundation Course and learn the basics.

To begin with, your interview preparations Enhance your Data Structures concepts with the Python DS Course.




My Personal Notes arrow_drop_up

I code for fun

If you like GeeksforGeeks and would like to contribute, you can also write an article using contribute.geeksforgeeks.org or mail your article to contribute@geeksforgeeks.org. See your article appearing on the GeeksforGeeks main page and help other Geeks.

Please Improve this article if you find anything incorrect by clicking on the "Improve Article" button below.


Article Tags :

Be the First to upvote.


Please write to us at contribute@geeksforgeeks.org to report any issue with the above content.