Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Hubs] Fail ingestion when older than the retention period #1372

Open
4 tasks
flanakin opened this issue Mar 7, 2025 · 0 comments
Open
4 tasks

[Hubs] Fail ingestion when older than the retention period #1372

flanakin opened this issue Mar 7, 2025 · 0 comments
Labels
Skill: Data factory Data Factory integration Skill: Documentation Documentation updates Tool: FinOps hubs Data pipeline solution Type: Feature 💎 Idea to improve the product

Comments

@flanakin
Copy link
Collaborator

flanakin commented Mar 7, 2025

📝 Scenario

As a FinOps hub admin, I need to fail data ingestion when a month that is older than the data retention period is attempted to be ingested in order to clarify why ingested data is not available

💎 Solution

In the ingestion_ExecuteETL pipeline, check if the ingested month is older than the final data retention. If older, then fail the pipeline run with a clear error that states the data is older than the data retention and add documentation to indicate that they need to update the final retention period.

📋 Tasks

Required tasks

Preview Give feedback

Stretch goals

Preview Give feedback

🙋‍♀️ Ask for the community

We could use your help:

  1. Please vote this issue up (👍) to prioritize it.
  2. Leave comments to help us solidify the vision.
@flanakin flanakin added Skill: Data factory Data Factory integration Skill: Documentation Documentation updates Tool: FinOps hubs Data pipeline solution Type: Feature 💎 Idea to improve the product labels Mar 7, 2025
@flanakin flanakin changed the title [Hubs] Fail ADX ingestion when the older than the configured data retention [Hubs] Fail ingestion when older than the retention period Mar 7, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Skill: Data factory Data Factory integration Skill: Documentation Documentation updates Tool: FinOps hubs Data pipeline solution Type: Feature 💎 Idea to improve the product
Projects
None yet
Development

No branches or pull requests

1 participant