Fix Celery worker crash on macOS due to unpicklable local function#62655
Conversation
|
cc @vincbeck @o-nikolas @potiuk — Would appreciate your review. This fixes a Celery worker crash on macOS where |
o-nikolas
left a comment
There was a problem hiding this comment.
Code looks reasonable to me. Any way we can add a test to avoid future regression?
Move bundle_cleanup_main() from a local function inside _run_stale_bundle_cleanup() to module-level _bundle_cleanup_main(). On macOS, multiprocessing uses spawn instead of fork, and local functions cannot be pickled for spawning across processes. Closes: apache#62512 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
df54e40 to
223df87
Compare
@o-nikolas I have added the test for that, regression. Could you please give me feedback for the test that I have added? If everything looks good to you, could you please give me a stamp for my PR? Thank you. |
|
@codex could you give the detailed review feedback for this PR? |
|
I made the exact same change a few days ago: #62528 . What a coincidence. |
dheerajturaga
left a comment
There was a problem hiding this comment.
Thanks! For catching this 😃
|
Nice! |
Summary
bundle_cleanup_main()from a local function inside_run_stale_bundle_cleanup()to module-level_bundle_cleanup_main(check_interval)spawninstead offork, and local functions cannot be pickled for spawning across processesserve_logs(module-level function used withProcess(target=serve_logs))Co-contributors : @codingrealitylabs @girlcoder-gaming
Test plan
Closes: #62512
🤖 Generated with Claude Code