Skip to content

Deferred trigger tasks seem to get stuck in Airflow 3.0.2 #52247

Closed
@haritamar

Description

@haritamar

Apache Airflow version

3.0.2

If "Other Airflow 2 version" selected, which one?

No response

What happened?

Hi, I'm currently in the process of migrating to Airflow 3.0.2.
As a part of our Airflow project, we are utilizing TriggerDagRunOperator with deferrable=True to trigger downstream DAGs, and we also use wait_for_completion=True since we measure SLAs using the duration of the main DAG.

We noticed that after some time the triggering tasks seem to get stuck at "deferred" even though the underlying DAG is completed:
Image

I believe what happens behind the scenes is that the async loop in the triggerer gets stuck, there are logs that seem to indicate that:
Image

What you think should happen instead?

All deferred tasks should be completed after the underlying DAGs completed (with an acceptable small delay)

How to reproduce

The example below causes the issue to reproduce for me.
The issue seems to be statistical - I need to trigger the DAG about 15 times in a row and it will eventually get stuck (that is, all the test_defer_issue_b DAGs will complete, but the triggering tasks will stay as deferred).

from datetime import datetime
from airflow.decorators import dag, task
from airflow.providers.standard.operators.trigger_dagrun import TriggerDagRunOperator

@dag(
    dag_id="test_defer_issue_a",
    start_date=datetime(2025, 6, 25),
    schedule=None,
)
def test_defer_issue_a():
    TriggerDagRunOperator(
        task_id="trigger_b_task",
        trigger_dag_id="test_defer_issue_b",
        deferrable=True,
        wait_for_completion=True
    )

test_defer_issue_a()


@dag(
    dag_id="test_defer_issue_b",
    start_date=datetime(2025, 6, 25),
    schedule=None,
)
def test_defer_issue_b():
    @task
    def test_task():
        print("test_task")

    test_task()

test_defer_issue_b()

Operating System

MacOS / Linux (both locally and in our dev envs)

Versions of Apache Airflow Providers

No response

Deployment

Official Apache Airflow Helm Chart

Deployment details

We use the official helm chart, version 1.16.0.

Anything else?

The issue might be related to this one, but I wasn't sure if it's the same thing since the behavior is a bit different in my case.
Feel free to merge if you feel otherwise.

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions