[WIP] - attempt to async/await conversion of common functions #1238
Conversation
auvipy
changed the title
|
What is the goal here for async/await? Creating parallel paths, or a single code base that would be used in either sync or async? |
we are still not 100% sure about that. but possibly we will try to make full async & figure out wheather we should keep something sync & might need to make a way out to keep async code work alongside sync one like django is doing. but may be the option one will be easier. we might also consider using trio+anyio+trio-asyncio fore the underlying aiolibs working with trio within celery |
|
Okay. I was under the impression that it would be something similar to selecting an alternative pool, like the eventlet option. This also could be used for something like celery/celery#6556 Just want to throw it out there as well, we were able to add trio support to Sanic without anyio and just a few small tweaks. I fully support the idea to allow multiple loops implementations, but before falling back to another dependency, it might be worth looking to see how deep the changes would need to be. Full async support might be the easiest to maintain in the long run, but also might be the biggest hit towards backwards compat. |
|
https://github.com/kai3341/celery-pool-asyncio this was a workaround type solution I initially thought about |
|
Abandoned or still viable? I guess the ultimate question is how much celery is allowed to change by embracing this. Has there been discussion or decision on this? Sorry if I'm asking a lot of questions that's old news, just trying to get up to speed. |
|
https://github.com/kai3341/celery-pool-asyncio this was a workaround type solution I
It's still a high priority for the next major release. not abandoned yet, as that approach might be good for keeping backward compat. you could check the package. but we are still not done with the full design yet. |
|
So where would be a good place to help? How can I be of assistance to you to get this rolling? The PRs for review are pretty small and it's not clear what direction to take since a compat layer, and full integration are two seperate things. |
My current suggestion would be to check the async-pool library codes & figure out the best design decision direction to true async/await capacity |
|
And I also have to do some more research & development |
fully async-await-based single code base. but not all code needs to be async. I will create a POC and CEP on this soon so tht we cn reach a concrete consensus for kombu & celery v6 |
sqlalchemy/sqlalchemy@5fb0138