Releasing GIL and mixing threads from C and Python

Releasing GIL in C extension:
Give a C extension code and one needs to concurrently execute it with other threads in the Python interpreter. For this Global Interpreter Lock (GIL) has to be released and reacquired.

Code #1 : Releasing and Reacquiring GIL by inserting following macros

filter_none

edit
close

play_arrow

link
brightness_4
code

#include "Python.h"
...
PyObject *pyfunc(PyObject *self, PyObject *args)
{
    ...
    Py_BEGIN_ALLOW_THREADS
      
    // Threaded C code. 
    // Must not use Python API functions
    ...
    Py_END_ALLOW_THREADS
    ...
    return result;
}

chevron_right


Mixing Threads from C and Python:
Considering a situation, that a given program involves a mix of C, Python and threads. But some of the given threads are created from C outside the control of the Python interpreter and also certain threads utilize functions in the Python C API.

What’s solution then?
If there is a mix of Python, C and threads, proper initialization and management of Python’s Global Interpreter Lock (GIL) is important. Given code below explains the situation, their code can be used anywhere (prior to the creation of threads) in the C program.

Code #2 :



filter_none

edit
close

play_arrow

link
brightness_4
code

#include <Python.h>
...
if (!PyEval_ThreadsInitialized())
{
    PyEval_InitThreads();
}
...

chevron_right


For any C involving Python objects or the Python C API, GIL needs to be acquired and released first. This can be performed using PyGILState_Ensure() and PyGILState_Release() as in the code given below. Every call to PyGILState_Ensure() must have a matching call to PyGILState_Release().

Code #3 :

filter_none

edit
close

play_arrow

link
brightness_4
code

...
// Make sure we own the GIL
// Use functions in the interpreter
PyGILState_STATE state = PyGILState_Ensure();
  
...
// Restore previous GIL state and return 
PyGILState_Release(state);
...

chevron_right


Note :
It is not so easy to have so much things going at once, involving a mix of a Python code, C code and their threads. To perform so, it is to be taken care that the interpreter is properly initialized and that C code involving the interpreter has the proper GIL management calls, it all should work.
Also, the PyGILState_Ensure() call does not immediately preempt or interrupt the interpreter. If other code is currently executing, this function will block until that code decides to release the Global Interpreter Lock (GIL).




My Personal Notes arrow_drop_up

Check out this Author's contributed articles.

If you like GeeksforGeeks and would like to contribute, you can also write an article using contribute.geeksforgeeks.org or mail your article to contribute@geeksforgeeks.org. See your article appearing on the GeeksforGeeks main page and help other Geeks.

Please Improve this article if you find anything incorrect by clicking on the "Improve Article" button below.


Article Tags :

Be the First to upvote.


Please write to us at contribute@geeksforgeeks.org to report any issue with the above content.