-
Notifications
You must be signed in to change notification settings - Fork 0
Add logic for dynamic data_source registering #17
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR adds dynamic data source registration functionality to handle unknown data sources encountered during message processing. Instead of failing when an unknown data source is encountered, the system now automatically registers it and retries the operation.
- Added automatic data source registration when
SzUnknownDataSourceErroroccurs - Enhanced error handling with message visibility restoration for failed operations
- Updated documentation and configuration to reflect the new dynamic capabilities
Reviewed Changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| middleware/consumer.py | Added register_data_source() function and enhanced error handling in message processing with dynamic registration |
| docker-compose.yaml | Updated PostgreSQL image and reduced pre-configured data sources to rely on dynamic registration |
| README.md | Reorganized documentation structure and added SQS queue management examples |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
Add logic to dynamically register new data_source names on the fly