Description
Apache Airflow version
3.0.2
If "Other Airflow 2 version" selected, which one?
No response
What happened?
I am in the process of upgrading from 2.10.3 to 3.0.2.
I have been relying on setting the env var AIRFLOW__CORE__DAG_DISCOVERY_SAFE_MODE=false
in order to parse dags from files that don't mention the words DAG & airflow. The configuration shows up as dag_discovery_safe_mode: false
in the airflow config page in both versions.
However it does not seem to be taking effect in version 3.0.2. I have confirmed that if I add # DAG airflow
to the top of the dag definition file the dag will appear in the airflow ui. This suggests that the dag parser is still in safe mode despite the configuration.
What you think should happen instead?
All files in the /dags
folder should be parsed, even if they do not mention DAG and airflow.
How to reproduce
- Set the env var
AIRFLOW__CORE__DAG_DISCOVERY_SAFE_MODE=false
- Add file
include/create_dag.py
:
from datetime import datetime
from airflow import DAG
from airflow.providers.standard.operators.empty import EmptyOperator
def create_dag():
with DAG(
"test_dag",
default_args={
"owner": "airflow",
"start_date": datetime(2021, 6, 18),
},
):
EmptyOperator(task_id="test")
- Add file
dags/test_dag.py
from include.dag_blueprints.create_dag import create_dag
create_dag()
This will not parse unless # airflow DAG is added to the top
Operating System
Debian
Versions of Apache Airflow Providers
No response
Deployment
Astronomer
Deployment details
No response
Anything else?
No response
Are you willing to submit PR?
- Yes I am willing to submit a PR!
Code of Conduct
- I agree to follow this project's Code of Conduct