Django Celery Gevent SynchronousOnlyOperation

[zN3utr4l]

I opened an issue about a library I’m using for my django project, here’s the link: Async DataBaseHandler (Sink?) Django · Issue #879 · Delgan/loguru · GitHub 2
Searching the internet I found a question on stackoverflow which describes exactly what happens: django - SynchronousOnlyOperation from celery task using gevent execution pool - Stack Overflow 1

commands:

celery==5.2.7
django==3.2.19

celery -A scheduler.celery beat -l INFO -f $path/celery/beat.log --uid=nobody --gid=nogroup --detach
celery -A scheduler.celery worker -l INFO -f $path/celery/worker.log --uid=nobody --gid=nogroup -O fair --pool=gevent --hostname=celery-workerAnalyser --autoscale=30,10 --without-heartbeat --without-gossip # --without-mingle

The main question is: But why isn’t this exception raised in 100% of cases but only in a minority of cases?

I figured out that django orm is not greenlet thread safe for gevent.

I already have a PG pooler, pgbouncer, so I would like to use its pooling, but I would like to understand if this library could help me with the error that celery throws when trying to query.

I already tried this comment: #61 (comment), but nothing changes.

[jneight]

How are you initializing gevent? using a monkey patch like the one described here: https://github.com/jneight/django-db-geventpool#patch-psycopg2

[zN3utr4l]

yes i tried this

Patch psycopg2
Before using the pool, psycopg2 must be patched with psycogreen, if you are using gunicorn webserver, a good place is the post_fork() function at the config file:

from psycogreen.gevent import patch_psycopg     # use this if you use gevent workers

def post_fork(server, worker):
    patch_psycopg()
    worker.log.info("Made Psycopg2 Green")

In this case I was using celery, I did the same thing with celery, celeryd-init and worker_init signals (https://docs.celeryq.dev/en/stable/userguide/signals.html#signals)