DAG execution fails with 'state mismatch' error when behind an NGINX reverse proxy

[gx8066liren]

Apache Airflow version

3.0.4

If "Other Airflow 2 version" selected, which one?

No response

What happened?

Description of the bug

When Airflow is running behind an NGINX reverse proxy, manually triggering a DAG from the web UI results in a task failure. The logs indicate a state mismatch error, where the LocalExecutor reports the task as failed, but the task instance's state remains queued. This issue seems to be related to the reverse proxy configuration, as the DAG runs correctly when accessing the webserver directly.

Environment

Airflow version: 3.0.4
OS: CentOS 7
Python version: 3.10
Executor: LocalExecutor
Deployment: airflow standalone running inside a Docker container.

Expected behavior

The DAG should be triggered and execute successfully, with task states updating correctly in the UI and logs.

Actual behavior

The DAG run fails. The task instance logs show the following state mismatch error:
Executor LocalExecutor(parallelism=32) reported that the task instance <TaskInstance: generate_hot_question_dag.generate_hot_question_task scheduled__2025-08-14T22:00:00+00:00 [queued]> finished with state failed, but the task instance's state attribute is queued. Learn more: https://airflow.apache.org/docs/apache-airflow/stable/troubleshooting.html#task-state-changed-externally

Relevant Configurations

NGINX Configuration:

map $http_upgrade $connection_upgrade {
    default Upgrade;
    ''      keep-alive;
}

location /scheduler/ {
    proxy_pass http://127.0.0.1:8080;
    proxy_http_version 1.1;
    proxy_set_header Host $http_host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection $connection_upgrade;
    proxy_redirect off;
}

Airflow Environment Variable:
AIRFLOW__API__BASE_URL=http://localhost/scheduler

Screenshots

What you think should happen instead?

No response

How to reproduce

1. Run Airflow using the airflow standalone command within a Docker container.
2. Configure an NGINX reverse proxy to forward requests from a subpath (e.g., /scheduler/) to the Airflow webserver (e.g., http://127.0.0.1:8080).
3. Set the base_url in the Airflow configuration to match the proxy path by setting the environment variable: AIRFLOW__API__BASE_URL=http://localhost/scheduler.
4. Access the Airflow UI via the NGINX proxy URL.
5. Manually trigger any DAG.

Operating System

CentOS 7

Versions of Apache Airflow Providers

apache-airflow-providers-common-compat==1.7.3
apache-airflow-providers-common-io==1.6.2
apache-airflow-providers-common-sql==1.27.5
apache-airflow-providers-smtp==2.2.0
apache-airflow-providers-standard==1.5.0

Deployment

Docker-Compose

Deployment details

No response

Anything else?

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[boring-cyborg[bot]]

Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.

[potiuk]

Wny do you use "airflow standalone" behind proxy (and for anything else than quick start and playing) ? It's clearly marked as not production ready. Is this also the case when airflow is deployed using one of regular production deployment? If so -please open an issue with details of deployment, but using airflow standalone for anything else than making a glimpse how airflow work is not supported - we discussed other options https://lists.apache.org/thread/cc7wfz9w8d1npopj72nlxnh52dpzbls9 but it was not well received (for then) - if you want to use it for anything more than running standalone - for playing with airflow - I suggest to start another discussion at devlist.

Converting to a discussion if more discussion is needed.

[kolfild26]

@gx8066liren you need to look at the worker logs. It most likely contains the information what's happening with a task

[U12358]

Today I played with similar issue, want to share my expirience. Airflow 3.1.7, K8s + Ingress.
Goal is configure dev system where http://localhost/airflow next to http://localhost/pgadmin, http://localhost/mlflow, ... everything shares one local host, one port and different subdir. Of course minimize resources.

Tried 3 variants:

1. Standalone, 100% unworking

AIRFLOW__API__BASE_UR=http://localhost:8080/airflow
(as result AIRFLOW__CORE__EXECUTION_API_SERVER_URL={AIRFLOW__API__BASE_UR}/execution)

3. Standalone, 50/50 either fully unworking or half broken

AIRFLOW__API__BASE_UR=http://localhost:8080/airflow
AIRFLOW__CORE__EXECUTION_API_SERVER_URL=http://localhost:8080/execution

or

5. Two containers, two api-servers, 100% working

1st container only api-server as Web
AIRFLOW__API__BASE_UR=http://localhost:8080/airflow

2nd container the second api-server as API + scheduler, triggerer, dag-proccessor
AIRFLOW__API__BASE_UR=http://localhost:8080/