airflow db clean crashes with ForeignKeyViolation when deleting dag_version in Airflow 3.1.8 #63703
Description
Apache Airflow version
3.1.8
If "Other Airflow 3 version" selected, which one?
No response
What happened?
The following command was executed:
airflow db clean --clean-before-timestamp 2026-01-01
The cleanup process successfully deletes rows from several tables (xcom, task_instance, dag_run, etc.), but fails when processing dag_version.
You have requested that we purge all data prior to 2026-01-01 00:00:00+00:00 for tables ['_xcom_archive', 'asset_event', 'callback_request', 'celery_taskmeta', 'celery_tasksetmeta', 'dag', 'dag_run', 'dag_version', 'deadline', 'import_error', 'job', 'log', 'sla_miss', 'task_instance', 'task_instance_history', 'task_reschedule', 'trigger', 'xcom'].
This is irreversible. Consider backing up the tables first and / or doing a dry run with option --dry-run.
Enter 'delete rows' (without quotes) to proceed.
delete rows
Checking table import_error
Found 0 rows meeting deletion criteria.
Checking table task_instance_history
Found 0 rows meeting deletion criteria.
Checking table xcom
Found 98 rows meeting deletion criteria.
Performing Delete...
Moving data to table _airflow_deleted__xcom__20260316091330
Finished Performing Delete
Checking table task_instance
Found 43572 rows meeting deletion criteria.
Performing Delete...
Moving data to table _airflow_deleted__task_instance__20260316091330
Finished Performing Delete
Checking table deadline
Found 0 rows meeting deletion criteria.
Checking table dag_run
Found 38877 rows meeting deletion criteria.
Performing Delete...
Moving data to table _airflow_deleted__dag_run__20260316091343
Finished Performing Delete
Checking table dag_version
Found 8 rows meeting deletion criteria.
Performing Delete...
Moving data to table _airflow_deleted__dag_version__20260316091351
Traceback (most recent call last):
File "/usr/local/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 1967, in _exec_single_context
self.dialect.do_execute(
~~~~~~~~~~~~~~~~~~~~~~~^
cursor, str_statement, effective_parameters, context
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
)
^
File "/usr/local/lib64/python3.13/site-packages/sqlalchemy/engine/default.py", line 952, in do_execute
cursor.execute(statement, parameters)
~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
psycopg2.errors.ForeignKeyViolation: update or delete on table "dag_version" violates foreign key constraint "task_instance_dag_version_id_fkey" on table "task_instance"
DETAIL: Key (id)=(019a828c-a2ec-7700-bad6-725dd65b53a1) is still referenced from table "task_instance".
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/local/sbin/airflow", line 10, in <module>
sys.exit(main())
~~~~^^
File "/usr/local/lib/python3.13/site-packages/airflow/__main__.py", line 55, in main
args.func(args)
~~~~~~~~~^^^^^^
File "/usr/local/lib/python3.13/site-packages/airflow/cli/cli_config.py", line 49, in command
return func(*args, **kwargs)
File "/usr/local/lib/python3.13/site-packages/airflow/utils/cli.py", line 114, in wrapper
return f(*args, **kwargs)
File "/usr/local/lib/python3.13/site-packages/airflow/utils/providers_configuration_loader.py", line 54, in wrapped_function
return func(*args, **kwargs)
File "/usr/local/lib/python3.13/site-packages/airflow/cli/commands/db_command.py", line 296, in cleanup_tables
run_cleanup(
~~~~~~~~~~~^
table_names=args.tables,
^^^^^^^^^^^^^^^^^^^^^^^^
...<5 lines>...
batch_size=args.batch_size,
^^^^^^^^^^^^^^^^^^^^^^^^^^^
)
^
File "/usr/local/lib/python3.13/site-packages/airflow/utils/session.py", line 100, in wrapper
return func(*args, session=session, **kwargs)
File "/usr/local/lib/python3.13/site-packages/airflow/utils/db_cleanup.py", line 541, in run_cleanup
_cleanup_table(
~~~~~~~~~~~~~~^
clean_before_timestamp=clean_before_timestamp,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
...<5 lines>...
batch_size=batch_size,
^^^^^^^^^^^^^^^^^^^^^^
)
^
File "/usr/local/lib/python3.13/site-packages/airflow/utils/db_cleanup.py", line 372, in _cleanup_table
_do_delete(
~~~~~~~~~~^
query=query,
^^^^^^^^^^^^
...<3 lines>...
batch_size=batch_size,
^^^^^^^^^^^^^^^^^^^^^^
)
^
File "/usr/local/lib/python3.13/site-packages/airflow/utils/db_cleanup.py", line 262, in _do_delete
raise e
File "/usr/local/lib/python3.13/site-packages/airflow/utils/db_cleanup.py", line 258, in _do_delete
session.execute(delete)
~~~~~~~~~~~~~~~^^^^^^^^
File "/usr/local/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2351, in execute
return self._execute_internal(
~~~~~~~~~~~~~~~~~~~~~~^
statement,
^^^^^^^^^^
...<4 lines>...
_add_event=_add_event,
^^^^^^^^^^^^^^^^^^^^^^
)
^
File "/usr/local/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2258, in _execute_internal
result = conn.execute(
statement, params or {}, execution_options=execution_options
)
File "/usr/local/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 1419, in execute
return meth(
self,
distilled_parameters,
execution_options or NO_OPTIONS,
)
File "/usr/local/lib64/python3.13/site-packages/sqlalchemy/sql/elements.py", line 527, in _execute_on_connection
return connection._execute_clauseelement(
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
self, distilled_params, execution_options
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
)
^
File "/usr/local/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 1641, in _execute_clauseelement
ret = self._execute_context(
dialect,
...<8 lines>...
cache_hit=cache_hit,
)
File "/usr/local/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 1846, in _execute_context
return self._exec_single_context(
~~~~~~~~~~~~~~~~~~~~~~~~~^
dialect, context, statement, parameters
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
)
^
File "/usr/local/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 1986, in _exec_single_context
self._handle_dbapi_exception(
~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
e, str_statement, effective_parameters, cursor, context
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
)
^
File "/usr/local/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 2363, in _handle_dbapi_exception
raise sqlalchemy_exception.with_traceback(exc_info[2]) from e
File "/usr/local/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 1967, in _exec_single_context
self.dialect.do_execute(
~~~~~~~~~~~~~~~~~~~~~~~^
cursor, str_statement, effective_parameters, context
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
)
^
File "/usr/local/lib64/python3.13/site-packages/sqlalchemy/engine/default.py", line 952, in do_execute
cursor.execute(statement, parameters)
~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
sqlalchemy.exc.IntegrityError: (psycopg2.errors.ForeignKeyViolation) update or delete on table "dag_version" violates foreign key constraint "task_instance_dag_version_id_fkey" on table "task_instance"
DETAIL: Key (id)=(019a828c-a2ec-7700-bad6-725dd65b53a1) is still referenced from table "task_instance".
[SQL: DELETE FROM dag_version USING _airflow_deleted__dag_version__20260316091351 WHERE dag_version.id = _airflow_deleted__dag_version__20260316091351.id]
(Background on this error at: https://sqlalche.me/e/20/gkpj)
What you think should happen instead?
airflow db clean should:
- delete rows in a safe dependency order, or
- skip dag_version rows that are still referenced, or
- ensure all referencing task_instance rows are removed first.
The command should not crash with a foreign key violation.
How to reproduce
airflow db clean --clean-before-timestamp 2026-01-01
Operating System
Fedora 5.3
Versions of Apache Airflow Providers
Latests
Deployment
Official Apache Airflow Helm Chart
Deployment details
No response
Anything else?
No response
Are you willing to submit PR?
- Yes I am willing to submit a PR!
Code of Conduct
- I agree to follow this project's Code of Conduct