Skip to content

Conversation

@spompea-cfa
Copy link
Contributor

@spompea-cfa spompea-cfa commented Sep 19, 2025

Add logic to dynamically register new data_source names on the fly

@spompea-cfa spompea-cfa requested a review from a team as a code owner September 19, 2025 19:23
@spompea-cfa spompea-cfa marked this pull request as draft September 19, 2025 19:23
@spompea-cfa spompea-cfa changed the title Add logic for dynamic data_source registering [WIP] Add logic for dynamic data_source registering Sep 19, 2025
@spompea-cfa spompea-cfa changed the title [WIP] Add logic for dynamic data_source registering Add logic for dynamic data_source registering Sep 22, 2025
@spompea-cfa spompea-cfa marked this pull request as ready for review September 22, 2025 18:47
@jamesiarmes jamesiarmes requested a review from Copilot September 24, 2025 16:22
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR adds dynamic data source registration functionality to handle unknown data sources encountered during message processing. Instead of failing when an unknown data source is encountered, the system now automatically registers it and retries the operation.

  • Added automatic data source registration when SzUnknownDataSourceError occurs
  • Enhanced error handling with message visibility restoration for failed operations
  • Updated documentation and configuration to reflect the new dynamic capabilities

Reviewed Changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

File Description
middleware/consumer.py Added register_data_source() function and enhanced error handling in message processing with dynamic registration
docker-compose.yaml Updated PostgreSQL image and reduced pre-configured data sources to rely on dynamic registration
README.md Reorganized documentation structure and added SQS queue management examples

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

@jamesiarmes jamesiarmes merged commit 0192fe0 into main Sep 24, 2025
9 checks passed
@jamesiarmes jamesiarmes deleted the add-data-source-dynamically branch September 24, 2025 18:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants