fix: wrap PythonExecute's blocking multiprocessing in run_in_executor to unblock event loop#1338
Open
programming-pupil wants to merge 1 commit into
Open
fix: wrap PythonExecute's blocking multiprocessing in run_in_executor to unblock event loop#1338programming-pupil wants to merge 1 commit into
programming-pupil wants to merge 1 commit into
Conversation
Author
|
Hi @appleboy @zhoupeng @Shellmode @cnJasonZ , just a gentle ping on this PR when you have some free cycles. All checks have passed. Please let me know if you need any additional context or changes from my end. Thanks! |
Author
|
@appleboy @zhoupeng @Shellmode @cnJasonZ Hi, I’m a big fan of OpenManus and would love to see it keep evolving. Since it’s been a while since the last update, I wanted to offer my help. Beyond this PR, I’m very much open to helping with issue triaging or ongoing maintenance if you’re looking for a co-maintainer. Let me know your thoughts! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
## Summary
PythonExecute.execute()is anasyncmethod but internally callsmultiprocessing.Process.start()+proc.join(timeout)synchronously.proc.join()is a blocking call that halts the entire asyncio event loop for up totimeoutseconds (default 5s), preventing all other concurrent coroutines (LLM calls, web fetches, other tool executions) from making progress.This PR wraps the blocking multiprocessing work in
asyncio.get_event_loop().run_in_executor()so it runs in a thread pool, keeping the event loop free.## Changes
app/tool/python_execute.py: Extracted the synchronousmultiprocessing.Manager+Process.joinblock into a private_run_in_process()method, and called it viarun_in_executor(None, ...)from the asyncexecute()method## Why this matters
In a typical OpenManus session, multiple tools may be active concurrently — the LLM is streaming, web searches are in flight, browser actions are pending. When
PythonExecuteblocks the event loop for 5 seconds (or longer with a custom timeout), all of these stall completely. This is especially painful when chained with other async tools in a planning flow.## Test plan
python -m py_compile app/tool/python_execute.pypassesawait PythonExecute().execute(code="print('hello')")returns{"observation": "hello\n", "success": True}await PythonExecute().execute(code="import time; time.sleep(10)", timeout=2)returns timeout error without blocking the event loop for 10sPythonExecuteis waiting for its subprocess