Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide (https://github.com/apache/airflow/blob/main/contributing-docs/README.rst)
Here are some useful points:

• Pay attention to the quality of your code (ruff, mypy and type annotations). Our pre-commits will help you with that.
• In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example DAG that shows how users should use it.
• Consider using Breeze environment for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
• Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
• Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
• Be sure to read the Airflow Coding style.
• Always keep your Pull Requests rebased, otherwise your build might fail due to changes not related to your commits.
  Apache Airflow is a community-driven project and together we are making it better 🚀.
  In case of doubts contact the developers at:
  Mailing List: [email protected]
  Slack: https://s.apache.org/airflow-slack

@jason810496

Thank you for bringing up the idea that the Execution API should also align with the Upsert behavior. :)
I agree with that, and I’ll update this PR to ensure the Execution API follows the same approach.

When you mentioned the Execution API, were you referring to the following path?
/airflow-core/src/airflow/api_fastapi/execution_api/routes/variables.py
It looks like this path is already using the same Variable.set.
If not, I’d appreciate it if you could point me to the correct path.

It looks like this path is already using the same Variable.set.

Oh yes, you are right! I misremembered — the Execution API indeed uses Variable.set from the model instead of TaskSDK.
We'll still need to wait for @ashb or @amoghrajesh to confirm whether the Upsert behavior is acceptable.

@jason810496

Thank you for the suggestion!

I've reviewed the behavior of Variable.set and confirmed that the existing test cases already cover the functionality for the key, val, and description parameters.

In addition, I've added a new test case, test_variable_set_update_existing, to explicitly verify the transition from the delsert to the upsert approach.

I wonder if it's better to use an actual upsert at the DB level when available https://docs.sqlalchemy.org/en/20/dialects/postgresql.html#insert-on-conflict-upsert (SQLIte has something similar in SQLA too. I don't know if mysql does)

Edit: Yup, it does https://docs.sqlalchemy.org/en/20/dialects/postgresql.html#insert-on-conflict-upsert

@ashb

Thank you very much for your feedback. :)
I'm already aware of the INSERT ... ON CONFLICT DO UPDATE (or the equivalent) syntax supported by PostgreSQL, MySQL, and SQLite.

However, since SQLAlchemy ORM does not offer a DB-agnostic way to express upserts using these features, I believe the current approach—checking whether the variable exists and then choosing between update or add—is the most universal and reliable solution at the ORM level.

In summary, while your suggestion to leverage a DB-level upsert is certainly valid, given the lack of a DB-agnostic upsert mechanism in SQLAlchemy ORM, I believe the current implementation is the most appropriate choice.

Additionally, it's technically possible to check the session's dialect first and then branch accordingly to use INSERT ... ON CONFLICT DO UPDATE (or the equivalent) with SQLAlchemy Core, depending on the specific database.

Would it be alright if I write it the way I explained above?

I’d appreciate your thoughts on this. Thank you!
@ashb

However, since SQLAlchemy ORM does not offer a DB-agnostic way to express upserts using these features, I believe the current approach—checking whether the variable exists and then choosing between update or add—is the most universal and reliable solution at the ORM level.

It is already a well-established pattern in Airflow to have if mysql/if postgres branches. -- espeically as all three supported database have support for ON CONFLICT UPDATE

@ashb

Thanks for your advice!

As you mentioned, I confirmed that dialect-specific branching pattern is already used in Airflow, and I’ve applied the same pattern to Variable.set.

Tested on PostgreSQL, MySQL, and SQLite.

I'd appreciate it if you could review the changes when you have a moment.

Thanks for your advice ! @jason810496

As you suggested, the code has been minimized:

• Applied dialect-specific insert functions and upsert clauses
• Consolidated the code wherever possible

Also tested on PostgreSQL, MySQL, and SQLite.

I'd appreciate it if you could provide any additional feedback !

cc. @ashb

Thanks, that was useful!
I simplified it using import_string and a mapping.

@jason810496

Thanks ! @jason810496

I'd appreciate it if you could provide any additional feedback. @ashb

Thank you!

I agree with all of your suggestions and have applied them accordingly.

Looking forward to the next steps for this PR after the 3.0.0 release.

@ashb

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for your contributions.

It seems that the issue occurred due to differences with the main branch since the last update in April. I've resolved it by merging the latest changes from the main branch.
Thank you for the advice.
@amoghrajesh, @uranusjr

Awesome work, congrats on your first merged pull request! You are invited to check our Issue Tracker for additional contributions.