DbtCloudRunJobOperator does not enforce execution_timeout semantics in Deferrable mode

[SameerMesiah97]

Apache Airflow Provider(s)

dbt-cloud

Versions of Apache Airflow Providers

apache-airflow-providers-dbt-cloud==4.6.4

Apache Airflow version

main

Operating System

Debian GNU/Linux 12 (bookworm)

Deployment

Other

Deployment details

No response

What happened

When using DbtCloudRunJobOperator in deferrable mode with execution_timeout set, the Airflow task times out but the underlying dbt Cloud job continues running.

In non-deferrable mode, exceeding execution_timeout causes both the task to fail and the dbt Cloud job to be cancelled. In deferrable mode, the task fails due to timeout, but the external job is not stopped, which can result in long-running or orphaned dbt Cloud jobs.

This creates inconsistent behavior between deferrable and non-deferrable execution modes.

What you think should happen instead

execution_timeout should be enforced consistently regardless of execution mode.

When a deferrable DbtCloudRunJobOperator exceeds execution_timeout:

• The Airflow task should fail due to execution timeout
• The associated dbt Cloud job should be cancelled

This ensures predictable timeout behavior and prevents leaked dbt Cloud jobs.

How to reproduce

1. Configure a dbt Cloud connection in Airflow.
2. Create a dbt Cloud job that runs longer than 30 seconds (the execution_timeout used in this reproduction).
3. Create the following DAG (replace <JOB_ID> wiht the ID of your DBT Cloud Job).

from airflow import DAG
from airflow.providers.dbt.cloud.operators.dbt import DbtCloudRunJobOperator
from datetime import datetime, timedelta

with DAG(
    dag_id="dbt_deferrable_execution_timeout_repro",
    start_date=datetime(2024, 1, 1),
    schedule=None,
    catchup=False,
) as dag:

    run_dbt = DbtCloudRunJobOperator(
        task_id="run_dbt",
        job_id=<JOB_ID>,
        wait_for_termination=True,
        deferrable=True,
        execution_timeout=timedelta(seconds=30),
    )

4. Trigger the DAG and wait for the task to exceed execution_timeout.

Observed Behavior

• The Airflow task fails due to execution timeout.
• The dbt Cloud job continues running and is not cancelled.

Anything else

This inconsistency makes execution_timeout unreliable for deferrable dbt Cloud jobs and can lead to unintended dbt Cloud resource usage. While deferrable operators execute via the triggerer rather than a worker process, this does not preclude enforcing execution_timeout semantics. The expectation is that task-level timeout behavior remains consistent across execution modes, even if the underlying enforcement mechanism differs..

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[SameerMesiah97]

@jroachgolf84

I would appreciate if you could assign this issue to me as this is meant to document the reproduction for PR #61472, which I have just opened. Also, if you are able to, could you please remove the 'good first issue' label to avoid inviting duplicate efforts?

[dylanschw]

Is this available to work on?

[SameerMesiah97]

Is this available to work on?

Thank you for your interest. But it has been assigned already with a PR currently being reviewed.

There are other issues on the board that you can have a look at if you want to contribute.

[potiuk]

@SameerMesiah97 We are unassigning you from this issue as part of our updated assignment policy.

This is not meant to discourage your contribution — quite the opposite! You are still very welcome to work on this issue and submit a PR for it. Simply comment that you are working on it and open a PR when ready.

We found that formal assignments were not working well, as they often prevented others from contributing when the assignee was not actively working on the issue.