Detailed Analysis on affects of Dynamic Typing and Concurrency on Python?
Is python too slow? The reality is, Python is an extremely slow programming language when we look at something like Java, C, and C++ it’s embarrassing how much faster they can do things than Python, in fact, specific algorithms and applications can actually do things about a hundred to two hundred times faster than the native Python language so in this article we’re going to discuss why that is and the different techniques that we can use as Python programmers to speed up and run more concurrent Python applications, now before we get too far it is worth noting that although Python lacks speed it makes that up in development and developing programs in Python is much faster, simpler and ends up costing less as there’s less labor involved with the project now this is a massive advantage in many cases you can actually write JAVA equivalent code in Python in about four to five times faster than you’d be able to do it in that language so this is something to consider?
Why is Python slow ?
The main reason this language (Python) is slow is not the global interpreter lock although that is a factor to the speed and the way that we can write faster Python programs that are not the reason fundamentally why the language is slow. The reason Python is slow is that it’s dynamically typed now we’re going to talk about this more in detail, but we want to give a comparison to a language like Java. Now in Java, everything is statically typed and this language is compiled before it runs, unlike Python that’s compiled at runtime through an interpreter. Now what happens in Java is when you write code, you need to define what type each of your variables is going to be, what type your methods and functions are going to be returning and you pretty much have to define exactly what everything’s going to be throughout your code. Now although this leads to much longer development times and takes a much longer time to write your code but what it does is increase efficiency when you are compiling, now the reason this actually works and the reason it works so much faster than Python code is because if you know the type that a specific variable or object is going to be, you can perform a ton of different optimizations and avoid performing a ton of different checks while you’re actually running the code because these checks are performed at compile time in Java essentially you can’t compile any Java code that hasn’t actual or even just like typed errors while you’re writing that code you are going to try to compile it and it would say like this type isn’t accurate, you can’t do this, you can’t compile it because it knows that when it comes to runtime that’s not going to work so essentially all of these checks that actually needs to be performed in Python when the code is running are performed beforehand and there’s just a ton of optimization done because of this statically typed length. Now one may ask a question like, Why doesn’t Python do this? Answer to this would be Python is dynamically typed which simply means that any variable can change its type and can change its value at any point in the program while it’s running which means that we can’t actually compile the entire program beforehand because we can’t do all of these checks at once because we don’t know what type these variables are going to be, they are going to change at runtime, different things are going to happen and because of that we can’t get all these optimizations that we might have in a lower level language like Java, C or C++ and that is kind of the fundamental reason the language is slow, this dynamic typing and any fast language is going to have a compiler that’s going to run through, it’s going to make sure that everything is good, it’s going to do all these checks before it actually ends up running the code at runtime where what happens in Python is all of your code is actually compiled and checked at runtime so rather than compiling it before and taking all that time beforehand while you’re running the code, many checks are happening to make sure that say this object is correct, these types are proper, everything is working the same.
Now the next thing to talk about is the lack of concurrency in Python. This is going to be the major kind of factor on speed, if you’re writing an application in Java, C, you can spread everything out throughout multiple threads which allows you to utilize all the cores of your CPU so to break this down in modern-day computing most of us have four core CPUs or higher and that allows us to run four tasks at the same time concurrently now with Python this isn’t possible. Python says, well for each interpreter we can have at most one thread running at a time and an thread is just some kind of operation that’s happening on the CPU core so that means even if we create many threads in our Python program we can only be using one CPU core while in a Java program or a C program could be using all eight or be using all four which will lead to 4X or 8X increase in speed, now we can get around this in Python by using multiprocessing, but there are some issues with that.
Why the Global Interpreter Lock exists in Python, why is that a feature of the language?
Well, the answer is simple, this goes back to the dynamic type of Python, so the way the memory is managed in Python, it’s not thread-safe now i.e., if two threads are trying to access one specific object in memory at the same time you’ll see some issues, essentially we can’t allow that to happen so what Python does is to have a global interpreter lock which means that only one thread can run at a time to prevent this from happening because we know that the major issue with Multiprocessing and Multithreading applications is that you have to deal with locking and sharing memory.
How can we speed up our Python code ?
One way to do this is to use C code as an extension to our Python Library. Now Python is built on top of C this is why sorting algorithms in Python will run much faster than if you write your native sort in Python. So if you need to create something that’s going to run very quickly in Python and you can’t use a different language, we can write that algorithm in C and import it into your Python code as an extension so you can run that code faster than if you just write it natively in Python.
- Python is mainly slow because of the way that the language is built i.e., because of the dynamic typing involved in Python we are not able to introduce a lot of optimizations in our compiling and interpreting that we have in other languages like Java.
- To increase speed of your Python code you can use some kind of threading and concurrency, you can use the multiprocessing module which will allow you to have multiple interpreters running at once and got past that global interpreter lock but you may run into some issues with shared and locking memory.
- Another way to speed up your Python code is that you can write some of your own C extensions for your Python code.