backfill

[bangbangDong]

Apache Airflow version

2.10.2

If "Other Airflow 2 version" selected, which one?

No response

What happened?

In normal run, this would be considered normal or skipped tasks, but in backfill it would be considered a failure. (I can't keep using the --continue-on-failures argument.)

What you think should happen instead?

No response

How to reproduce

Run the history task with backfill and build the task with task.virtualenv, venv_cache_path

Operating System

ubuntu

Versions of Apache Airflow Providers

No response

Deployment

Official Apache Airflow Helm Chart

Deployment details

No response

Anything else?

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[boring-cyborg[bot]]

Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.

[potiuk]

I am not sure what is the ask. Look a bit like you missed that you should spend a bit more time to explain what you think is wrong and what you think should happend @bangbangDong. I converted it into a discussion, but if you think that you would like to have some action as result of you report, spending a bit more of time to explain it would be a good idea, so I encourage you to do so.

[bangbangDong]

I just re-executed backfill, and all of my tasks look like this
@task
def my_task():
pass

[bangbangDong]

The task should be the same as the previous one, but it is marked as success to enable the task to run properly.

[bangbangDong]

In addition, another problem occurred when I was trying to backfill. In the process of backfill, all tasks in the next task have finished running, but its state is always running, which makes it impossible for backfill to continue to run. Moreover, this state seems to never stop. Sometimes it ran for days until I got him deleted

The above example is normal because I set --continue-on-failures in backfill.

However, in the same case, this task will remain in the running state and never stop. Even if all its tasks are displayed as success, the dag will still remain in the running state. Even if I set dagrun_timeout, this will not affect the situation.

[bangbangDong]

ps: My dag is set to max_active_runs=1

[potiuk]

No idea - but maybe someone will take a look at your report and will be able to understand it (there are some inconsistencies - in one place you mention python task.virtualenv with cache , in another you say you are using "@task" and you use wording that makes it hard to understand "task never stop"?.

Also it misses information and command you used from backfill and logs from backfill - because there likely you would see some errors (I am wildly guessing now).

Likely providing information like pasting the DAG you use, logs from the backfill command you use might help somoene to understand and replicate what you try to do (I am not expert in this part, so I can only advice what more information you can providde and request some consistency - hopefully if you provide those, maybe someone will be able to help.