Another person stuck trying to embed python (call into python) from threads.

[djacobow]

Hi there. I have an app in C++ that uses a lot of threads, and for testing and rapid development, it would be really nice for them to be able to call into python implementations of various things rather than implementing them all in C++.

I understand that because of the GIL, effective python execution will be through a single thread, or perhaps more accurately, "as if" through a single thread, but I'm still struggling to get it to work at all. What I think I have observed is that, no matter what I do with gil_scoped_acquire and gil_scoped_release, the only thread that can successfully call into python is the one that holds scoped_interpreter.

As a result, I've had to write a serializing wrapper (not shown) that all my threads call into that pushes python function calls and their arguments into a thread-safe queue, and the "python thread" runs a pop-execute loop fires a condition variable to notify the original calling thread of the result. This works, but I feel like it should be unnecessary, and is, strictly speaking, worse than even normal python threads since it cannot release the gil until completion of the function rather than real python that can swap threads when blocked on io.

Is there a better way to do this? I'd like to not have my serializer since I am told that the GIL is going away and I'd prefer that my code continue to be serialized when it does.

Here's a simplified example of what I tried first:

class naive_c {
  public:
    naive_c() : interp_(py::scoped_interpreter()) {
        py::exec("print('constructor')");
    }

    void bang() {
        std::thread t1([]() { py::exec("print('bang t1')"); });
        std::thread t2([]() { py::exec("print('bang t2')"); });
        t1.join();
        t2.join();
    };

  private:
    py::scoped_interpreter interp_;
};

int main(int ac, char *av[]) {
    auto n = std::make_unique<naive_c>();
    n->bang();
};

This crashes. I naively thought py::exec would automatically take/release the GIL as required, but apparently nope.

Then I tried this, and a bunch of variations:

class less_naive_c {
  public:
    less_naive_c() : interp_(py::scoped_interpreter()) {
        py::exec("print('constructor')");
        py::gil_scoped_release r {}; 
    }   

    void bang() {
        std::thread t1([]() {
            py::gil_scoped_acquire a {}; 
            py::exec("print('bang t1')");
        });
        std::thread t2([]() {
            py::gil_scoped_acquire a {}; 
            py::exec("print('bang t2')");
        });
        t1.join();
        t2.join();
    };  

  private:
    py::scoped_interpreter interp_;
};

int main(int ac, char *av[]) {
    auto n = std::make_unique<less_naive_c>();
    n->bang();
};

This does not crash, but it deadlocks.

Finally, the code below crashes, too, but after running the thread. I think what is happening is that in thread t1 the interpreter is created and when it exits it is not destroyed because the object holds it. When code (from the main thread) tries to destroy the object at the end of the program, it crashes because it's not the same thread.

class just_one_c {
  public:
    just_one_c() {}

    void bang() {
        std::thread t1([this]() {
            interp_ = std::make_unique<py::scoped_interpreter>();
            py::exec("print('bang t1')");
        });
        t1.join();
    }

  private:
    std::unique_ptr<py::scoped_interpreter> interp_;
};

I guess the last one is not unexpected.

I'm just curious if there is a better way to do this. Ideally, I'd like to just call into pybind::exec without any extra work, and it would be pybind that waits for the GIL, killing my thread parallelism, but at least the program runs.

Am I missing something?

Using:
Linux Ubuntu 24.04.1
clang 18.1.3
pybind 11 2.13.16
python 3.12.3

Best,
Dave J

[petersteneteg]

HI

I'm doing the same thing, and was also very confused about how to get this to work. But at least have found a working configuration,
I start of by calling

py::initialize_interpreter(&config);

The config, is optional...

That will initialize python and "capture" gil.
Then I do

PyThreadState* tstate = PyEval_SaveThread();

This will "release" the gil, and any other thread (or this one) can now capture the gil again using

const py::gil_scoped_acquire  guard{};

You now have to do this in all thread before any interaction with the interpreter, or there will be crash usually. This is also true for the thread that initialized the interpreter. Some pybind11 wrapper will automatically do this, for example calling a std::function that is assigned from python will automatically acquire the GIL, but generally you have to do it your self. It is fine to acquire it multiple times from the same thread as long as you release it the same number of times...

And in the end I restore the thread state before finalizing the interpreter

PyEval_RestoreThread(tstate);
py::finalize_interpreter();

I will leave it up to someone else to say if this is "correct" or not, but It works great for me.

[djacobow]

Thanks for engaging, Peter!

I tried your suggestion and it worked!

Here is a trivial working program. I wish the docs had something like this!

#include <format>
#include <thread>
#include <vector>

#include "pybind11/embed.h"

namespace py = pybind11;

void acquire_and_release() {
    py::gil_scoped_acquire g{};
};

void call_in_threads(uint32_t count) {
    std::vector<std::unique_ptr<std::thread>> tv;
    for (uint32_t i=0; i<count; i++) {
        tv.push_back(std::make_unique<std::thread>(
            [i]() {
                py::gil_scoped_acquire g{};
                py::exec(std::format("print('Hello from thread:{}')", i));
            }
        ));
    }
    for (auto &t : tv) {
        t->join();
    }

};

int main(int ac, char *av[]) {
    py::initialize_interpreter();
    PyThreadState *tstate = PyEval_SaveThread();

    acquire_and_release();
    call_in_threads(50);

    PyEval_RestoreThread(tstate);
    py::finalize_interpreter();
    return 0;
};

Note that I found that the acquire_and_release() function is required. You have to not only initialize the interpreter and call SaveThread, but you have to actually claim (and release) the gil from main before using it in threads. I don't understand it, but at least it seems to work. I hope someone who knows pybind11 internals will comment.

[petersteneteg]

No problem, just happy to shared that I got it working. I myself got clues from how pytorch does it :)

Nice find about the gil_scoped_acquire in the middle there. I have those too, but did not know that is was required, very odd..

I agree that the docs are very poor in this area. They mostly focus on the non embedded use cases it seems..
Would be great if someone could "confirm" if this is a "proper" way of doing this :)