Fix TriggerDagRunOperator deferring when wait_for_completion=False

[nathadfield]

Summary

Closes #60049

The bug was in the task runner's DagRunTriggerException handler, which checked the deferrable flag before checking wait_for_completion. This caused tasks with deferrable=True to incorrectly enter DEFERRED state even in fire-and-forget mode (wait_for_completion=False).

Root Cause

In task_runner.py, the handler for DagRunTriggerException was structured like:

if drte.deferrable:
    # Create and return TaskDeferred
    ...
if drte.wait_for_completion:
    # Wait logic
    ...

This meant that tasks would defer based solely on the deferrable flag, ignoring wait_for_completion.

Fix

Nested the deferrable check inside the wait_for_completion check:

if drte.wait_for_completion:
    if drte.deferrable:
        # Create and return TaskDeferred
        ...

Now tasks only defer when BOTH deferrable=True AND wait_for_completion=True.

Changes

• task-sdk/src/airflow/sdk/execution_time/task_runner.py: Fixed the exception handler logic
• task-sdk/tests/task_sdk/execution_time/test_task_runner.py: Updated test expectations to validate correct behavior

[nathadfield]

@kaxil @ashb Would either of you be able to take a look at this?

[uranusjr]

Is it ever useful for wait_for_completion=True to enter defer mode? Since it is fire-and-forget, there’s nothing to do after the run has been triggered successfully, and the task can simply end. Maybe it’d be a good idea to just ignore deferrable?

[nathadfield]

@uranusjr Setting wait_for_completion=True means that the task doesn't end until the DAG that was triggered finishes so, yes, deferring the task does makes sense in that context. The issue is when deferrable=True AND wait_for_completion=False does the task get stuck in deferred.

[uranusjr]

Sorry I mistyped. I meant deferrable=True can perhaps be ignored when wait_for_completion=False since there’s nothing to resume to anyway?

[nathadfield]

@uranusjr Yes, that's exactly the change I am proposing! Initially I modified the operators __init__ but looking more deeply I discovered that it could be resolved in the DagRunTriggerException instead.

[uranusjr]

Logically no problem. I’d recommend adding a log message (warning or at leat info) when deferrable=True is ignored so the user has better idea what happened.

[nathadfield]

Logically no problem. I’d recommend adding a log message (warning or at leat info) when deferrable=True is ignored so the user has better idea what happened.

Ok, will do!

[nathadfield]

@uranusjr Thanks for approving!

[github-actions]

Backport failed to create: v3-1-test. View the failure log Run details

Status	Branch	Result
❌	v3-1-test

You can attempt to backport this manually by running:

cherry_picker 2daadf4 v3-1-test

This should apply the commit to the v3-1-test branch and leave the commit in conflict state marking
the files that need manual conflict resolution.

After you have resolved the conflicts, you can continue the backport process by running:

cherry_picker --continue

[kaxil]

Good catch @nathadfield