-
Notifications
You must be signed in to change notification settings - Fork 0
Add handling for long-running Senzing calls #29
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
49cac1c to
9c2b0fa
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
Adds timeout handling for long-running Senzing calls to prevent indefinite blocking and implements graceful cleanup logic for process termination signals.
- Introduces configurable timeout mechanism using SIGALRM to abandon Senzing calls that exceed the specified timeout
- Updates SQS message visibility timeout to match the Senzing call timeout for better queue management
- Adds signal handlers for SIGINT/SIGTERM to restore message visibility before process termination
Reviewed Changes
Copilot reviewed 4 out of 5 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| middleware/timeout_handling.py | New module implementing alarm-based timeout mechanism with custom exception |
| middleware/redoer.py | Adds timeout wrapper around process_redo_record calls with error handling |
| middleware/consumer.py | Adds timeout wrapper around add_record calls and signal handlers for graceful shutdown |
| README.md | Documents the new SZ_CALL_TIMEOUT_SECONDS environment variable |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
jamesiarmes
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good overall. I like seeing the SIGINT/SIGTERM handling.
Add an environment variable SZ_CALL_TIMEOUT_SECONDS and supporting timeout logic so
that Senzing calls that continue past the timeout are abandoned (and logged as errors).
(Note that a Senzing call cannot ever actually be cancelled.) In the future we may want to
revisit / reevaluate this approach.
Also add logic to restore visibility (in SQS) to the currently-in-light message if/when the Consumer
receives a SIGINT / SIGTERM signal.