Description

Following the enhancements in PR #52228, which aligns the Databricks workflow provider plugin to support Airflow 3 and introduces XCom-based storage for extra links, this issue tracks the remaining work to reintroduce and redesign the repair functionality for compatibility with Airflow 3.

Background:
Airflow 3 blocks constructs like @provide_session and direct database access, moving toward a secure and API-driven model. The current repair flow is incompatible with this model for the following reasons:

1. Extra links (e.g., “See Databricks Job Run”) are now stored in XCom at task execution time. However, Databricks’ repair API requires knowing which specific tasks to repair at the time of invocation, which is only possible after failures occur — creating a timing mismatch.
2. The current repair implementation relies on a Flask AppBuilder (FAB) view exposed via the /repair_databricks_job endpoint, which does not align with the future direction of Airflow extensibility using FastAPI based endpoint.

Goals:

1. Re-evaluate the approach to repair in Airflow 3, given the new architecture for link computation and the runtime ambiguity around which tasks will need repair.
2. Explore how the repair functionality can be implemented as a FastAPI-based endpoint, aligning with Airflow’s evolving API architecture.
3. Investigate authentication and authorization patterns for securing a FastAPI repair endpoint.
4. Consider deferring the repair redesign until after the plugin extensibility improvements planned for Airflow 3.1 are available, to ensure better alignment with long-term architecture. This may offer a cleaner and more sustainable path for integrating custom endpoints and plugin-level APIs.

I’d be glad if someone from the community is interested in picking this up or would like to collaborate with me on it — especially if they’re familiar with the topic and Airflow 3.

Use case/motivation

No response

Related issues

related: #40587

Are you willing to submit a PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct