Skip to content

Provider Migration: Update yandex provider for Airflow 3.0 compatibility #52422

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 13 commits into from
Jun 30, 2025

Conversation

kevinhongzl
Copy link
Contributor

Part of #52378


^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

@kevinhongzl kevinhongzl force-pushed the update-yandex-provider-for-af3-compat branch from c3f2aa8 to fec4e23 Compare June 28, 2025 20:33
@kevinhongzl
Copy link
Contributor Author

Hi @potiuk @kaxil. I might need your help on yandex backward compat tests.

Previously, when we called persist, the function call chain was persist() -> BaseOperator.push_xcom() -> TI.push_xcom(). A redundant arg (execution_date) was added during the BashOperator Xcom push.

However, in our current persist(), we push Xcoms directly through a task instance without adding execution_date. This means the rewrite will change how persist() behaves in AF2 and its outputs will no longer match the expected output of the original test cases.

Given this, should I just modify the AF2 test cases, or is there a better way to handle this? Thanks.

@potiuk
Copy link
Member

potiuk commented Jun 29, 2025

Look at similar tests in google. I just fixed them a day ago or so: #52398 -> generally when in doubt - look for similar patterms in other providers.

@kaxil
Copy link
Member

kaxil commented Jun 30, 2025

One pre-commit is failing:

Update providers build files................................................Failed
- hook id: update-providers-build-files
- files were modified by this hook


--
diff --git a/providers/yandex/pyproject.toml b/providers/yandex/pyproject.toml
index 1eb44bd..876f8ad 100644
--- a/providers/yandex/pyproject.toml
+++ b/providers/yandex/pyproject.toml
@@ -73,7 +73,6 @@ dev = [
     "apache-airflow",
     "apache-airflow-task-sdk",
     "apache-airflow-devel-common",
-    "apache-airflow-providers-common-compat",
     # Additional devel dependencies (do not remove this line and add extra development dependencies)
     "responses>=0.25.0",
 ]

@potiuk
Copy link
Member

potiuk commented Jun 30, 2025

Yep. No more common.compat dependency - should be removed :)

@kaxil kaxil merged commit 3316407 into apache:main Jun 30, 2025
70 checks passed
@kevinhongzl
Copy link
Contributor Author

Thanks @potiuk @kaxil for your help and review on this PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants