Skip to content

dag_discovery_safe_mode broken in airflow 3 #52413

Open
@NilsJPWerner

Description

@NilsJPWerner

Apache Airflow version

3.0.2

If "Other Airflow 2 version" selected, which one?

No response

What happened?

I am in the process of upgrading from 2.10.3 to 3.0.2.

I have been relying on setting the env var AIRFLOW__CORE__DAG_DISCOVERY_SAFE_MODE=false in order to parse dags from files that don't mention the words DAG & airflow. The configuration shows up as dag_discovery_safe_mode: false in the airflow config page in both versions.

However it does not seem to be taking effect in version 3.0.2. I have confirmed that if I add # DAG airflow to the top of the dag definition file the dag will appear in the airflow ui. This suggests that the dag parser is still in safe mode despite the configuration.

What you think should happen instead?

All files in the /dags folder should be parsed, even if they do not mention DAG and airflow.

How to reproduce

  • Set the env var AIRFLOW__CORE__DAG_DISCOVERY_SAFE_MODE=false
  • Add file include/create_dag.py:
from datetime import datetime

from airflow import DAG
from airflow.providers.standard.operators.empty import EmptyOperator


def create_dag():
    with DAG(
        "test_dag",
        default_args={
            "owner": "airflow",
            "start_date": datetime(2021, 6, 18),
        },
    ):
        EmptyOperator(task_id="test")
  • Add file dags/test_dag.py
from include.dag_blueprints.create_dag import create_dag

create_dag()

This will not parse unless # airflow DAG is added to the top

Operating System

Debian

Versions of Apache Airflow Providers

No response

Deployment

Astronomer

Deployment details

No response

Anything else?

No response

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:corekind:bugThis is a clearly a bugneeds-triagelabel for new issues that we didn't triage yet

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions