Skip to content

Conversation

@steveahnahn
Copy link
Contributor

Problem

Manual triggers of scheduled Dags could generate the same run_id repeatedly (often aligned to the schedule boundary). Triggering the same Dag twice would then fail with a 409 Conflict due to a duplicate run_id

Solution

When a manual trigger provides a logical_date, Airflow still infers the correct data_interval for the run, but it no longer rewrites run_after to the scheduled data_interval.end. This keeps manual trigger run_ids based on the actual trigger time making them unique across repeated manual triggers.

  • Added a regression test that creates a scheduled Dag, triggers it twice, and asserts the returned dag_run_id values are different:

Manual testing in breeze triggering a daily scheduled dag via manual trigger runids:
scheduled__2025-12-15T00:00:00+00:00
manual__2025-12-15T04:16:39.584387+00:00
manual__2025-12-15T04:16:50.627593+00:00
manual__2025-12-15T04:18:07.769612+00:00

Related

Fixes #59342

@boring-cyborg boring-cyborg bot added the area:API Airflow's REST/HTTP API label Dec 15, 2025
@steveahnahn steveahnahn changed the title run id no longer collides in manual runs Unique run_id across manually triggered Dags with schedules Dec 15, 2025
@steveahnahn steveahnahn force-pushed the fix-colliding-run-ids-manual-runs branch from b530a1a to a2f862b Compare December 16, 2025 00:55
Copy link
Contributor

@bugraoz93 bugraoz93 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks Steve, changes looks good! Maybe we should make run_after unique rather than fallback to logicaldate. I remember this was an intentional change. I maybe missing something though

cc: @uranusjr

@pierrejeambrun pierrejeambrun added the backport-to-v3-1-test Mark PR with this label to backport to v3-1-test branch label Dec 18, 2025
@pierrejeambrun pierrejeambrun added this to the Airflow 3.1.6 milestone Dec 18, 2025
Copy link
Member

@pierrejeambrun pierrejeambrun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, just a question.

Copy link
Member

@pierrejeambrun pierrejeambrun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@pierrejeambrun pierrejeambrun merged commit 13f3e26 into apache:main Jan 13, 2026
126 checks passed
github-actions bot pushed a commit that referenced this pull request Jan 13, 2026
…es (#59477)

* run id no longer collides in manual runs

* ruff format;
(cherry picked from commit 13f3e26)

Co-authored-by: Steve Ahn <steveahnahn@g.ucla.edu>
@github-actions
Copy link

Backport successfully created: v3-1-test

Status Branch Result
v3-1-test PR Link

github-actions bot pushed a commit to aws-mwaa/upstream-to-airflow that referenced this pull request Jan 13, 2026
…es (apache#59477)

* run id no longer collides in manual runs

* ruff format;
(cherry picked from commit 13f3e26)

Co-authored-by: Steve Ahn <steveahnahn@g.ucla.edu>
pierrejeambrun pushed a commit that referenced this pull request Jan 14, 2026
…es (#59477) (#60468)

* run id no longer collides in manual runs

* ruff format;
(cherry picked from commit 13f3e26)

Co-authored-by: Steve Ahn <steveahnahn@g.ucla.edu>
ephraimbuddy pushed a commit that referenced this pull request Jan 14, 2026
…es (#59477) (#60468)

* run id no longer collides in manual runs

* ruff format;
(cherry picked from commit 13f3e26)

Co-authored-by: Steve Ahn <steveahnahn@g.ucla.edu>
ephraimbuddy pushed a commit that referenced this pull request Jan 16, 2026
…es (#59477) (#60468)

* run id no longer collides in manual runs

* ruff format;
(cherry picked from commit 13f3e26)

Co-authored-by: Steve Ahn <steveahnahn@g.ucla.edu>
jason810496 pushed a commit to jason810496/airflow that referenced this pull request Jan 22, 2026
…9477)

* run id no longer collides in manual runs

* ruff format;
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:API Airflow's REST/HTTP API backport-to-v3-1-test Mark PR with this label to backport to v3-1-test branch

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Conflicting manually triggered DAG's run IDs

4 participants