Description
Apache Airflow version
3.0.2
If "Other Airflow 2 version" selected, which one?
No response
What happened?
Hi, I'm currently in the process of migrating to Airflow 3.0.2.
As a part of our Airflow project, we are utilizing TriggerDagRunOperator
with deferrable=True
to trigger downstream DAGs, and we also use wait_for_completion=True
since we measure SLAs using the duration of the main DAG.
We noticed that after some time the triggering tasks seem to get stuck at "deferred" even though the underlying DAG is completed:
I believe what happens behind the scenes is that the async loop in the triggerer gets stuck, there are logs that seem to indicate that:
What you think should happen instead?
All deferred tasks should be completed after the underlying DAGs completed (with an acceptable small delay)
How to reproduce
The example below causes the issue to reproduce for me.
The issue seems to be statistical - I need to trigger the DAG about 15 times in a row and it will eventually get stuck (that is, all the test_defer_issue_b
DAGs will complete, but the triggering tasks will stay as deferred
).
from datetime import datetime
from airflow.decorators import dag, task
from airflow.providers.standard.operators.trigger_dagrun import TriggerDagRunOperator
@dag(
dag_id="test_defer_issue_a",
start_date=datetime(2025, 6, 25),
schedule=None,
)
def test_defer_issue_a():
TriggerDagRunOperator(
task_id="trigger_b_task",
trigger_dag_id="test_defer_issue_b",
deferrable=True,
wait_for_completion=True
)
test_defer_issue_a()
@dag(
dag_id="test_defer_issue_b",
start_date=datetime(2025, 6, 25),
schedule=None,
)
def test_defer_issue_b():
@task
def test_task():
print("test_task")
test_task()
test_defer_issue_b()
Operating System
MacOS / Linux (both locally and in our dev envs)
Versions of Apache Airflow Providers
No response
Deployment
Official Apache Airflow Helm Chart
Deployment details
We use the official helm chart, version 1.16.0.
Anything else?
The issue might be related to this one, but I wasn't sure if it's the same thing since the behavior is a bit different in my case.
Feel free to merge if you feel otherwise.
Are you willing to submit PR?
- Yes I am willing to submit a PR!
Code of Conduct
- I agree to follow this project's Code of Conduct