Feature(call-back): Enhance DAG-level callback context with enriched metadata and test coverage #51949
Conversation
|
Working on it. |
…metadata and test coverage
Co-authored-by: Tzu-ping Chung <[email protected]>
c1185a1 to
9493126
Compare
|
Hey @uranusjr, We can't use context.update(task_instance=ti) since TypedDict.update() doesn't support keyword arguments — mypy throws an error:
So I've updated it to use a standard dict instead. Please re-review the changes. Thanks in advance. |
| ``{{ mark_success_url }}`` str | None |URL to mark the DAG run as successful in the Airflow UI. | ||
| ``{{ log_url }}`` str | None |URL to the log for the current DAG run or task instance. | ||
| ``{{ dag_run_url }}`` str | None |URL to the DAG run details page in the Airflow UI. | ||
| ``{{ end_date }}`` DateTime | None |The end date/time of the DAG run. | ||
| ``{{ max_tries }}`` int | None |The maximum number of tries for the task instance. |
There was a problem hiding this comment.
I think this should be removed now they are not being added
| success_tis.sort(key=lambda x: x.end_date, reverse=True) | ||
| last_relevant_ti = success_tis[0] if success_tis else None |
There was a problem hiding this comment.
This can be something like
last_relevant_ti = max(success_tis, ...) if success_tis else Noneinstead.
I kind of wonder if we can even avoid building the list at all.
Also is simply sorting by end_date correct? Especially with trigger_rule, the last success/failed ti might not necessarily be the ti that causes the dag run to be marked as success/failed. Can you check the logic in 2.x to see how the ti is selected?
|
|
||
| return prefix + ">" |
| "dag_run_url", | ||
| "end_date", | ||
| "log_url", | ||
| "mark_success_url", | ||
| "max_tries", |
Summary
This PR enriches the context passed to DAG-level callbacks (on_success_callback and on_failure_callback) to better align with the structure and metadata typically available in task-level callbacks. While this logic is currently triggered via execute_callbacks=True during tests, it is structured to reflect what should be expected in production use cases as well.
Related
Closes: [#51402]