Detailed explanation of thread mixing in C and Python

concern

You have a program that uses a mix of C, Python, and threads. Some of the threads are created in C and are beyond the control of the Python interpreter. And some threads also use functions from the Python C API.

prescription

If you want to mix C, Python and threads, you need to make sure that you initialize and manage Python's Global Interpreter Lock (GIL) correctly. To do this, put the following code into your C code and make sure it is called before any threads are created.

#include <>
 ...
 if (!PyEval_ThreadsInitialized()) {
  PyEval_InitThreads();
 }
 ...

For any C code that calls a Python object or the Python C API, make sure you've properly acquired and released the GIL in the first place. This can be done using thePyGILState_Ensure() respond in singingPyGILState_Release() to do so, as shown below:

...
/* Make sure we own the GIL */
PyGILState_STATE state = PyGILState_Ensure();

/* Use functions in the interpreter */
...
/* Restore previous GIL state and return */
PyGILState_Release(state);
...

Each time thePyGILState_Ensure() All have to be called accordinglyPyGILState_Release() .

talk over

In high-level programs involving C and Python, it's quite common for a lot of things to be done together-- possibly a mix of C, Python, C threads, and Python threads. As long as you make sure that the interpreter is properly initialized and that the C code involving the interpreter performs proper GIL management, there shouldn't be much of a problem.

One thing to note is that calling thePyGILState_Ensure() It does not immediately preempt or interrupt the interpreter. If some other code is executing, the function is interrupted until that executing code releases the GIL. Internally, the interpreter performs periodic thread switches, so that if some other thread is executing, the caller will eventually be able to run (although it may have to wait a while first).

Above is a detailed explanation of thread mixing in C and Python in detail, more information about C and Python thread mixing please pay attention to my other related articles!