docs(task-sdk): reorganize landing page into six core sections #52196
Conversation
092c4dc to
eb70f52
Compare
eb70f52 to
88ef0d3
Compare
…Interface, DAG Authoring, Examples, Concepts, API Reference)
88ef0d3 to
fd59275
Compare
| specific language governing permissions and limitations | ||
| under the License. | ||
|
|
||
| Concepts |
There was a problem hiding this comment.
I don’t think these concepts belong in the task sdk documentation.
These docs should focus solely on the DAG author and interfaces relevant to them.
Broader architectural or system level concepts are better suited for the core Airflow docs.
There was a problem hiding this comment.
I can move them to airflow-core. But I wanted to add a separate page for concepts, as it is important in the task-sdk as well.
There was a problem hiding this comment.
Not really, I do not think it matters for task sdk
There was a problem hiding this comment.
It is not directly part of task-sdk, but Airflow has it to make task-sdk work. For example, concepts of Supervisor & Task Runner. Should I move them to the airflow core then?
There was a problem hiding this comment.
Also, the concept of metadata access
There was a problem hiding this comment.
I do not have a strong opinion on this, will wait on others for it.
Supervisor + task runner can probably stay but we should certainly not document API server here.
This PR reorganises and updates the content for task-sdk:
Landing page (task-sdk/docs/index.rst):
1. Introduction & Getting Started
2. Public Interface
3. DAG Authoring Enhancements
4. Example DAG References
5. Concepts
7. API References
Add concepts:
– Explains the service-oriented Task Execution API Server
– Compares Airflow 2.x vs 3.x architectures
– Details task lifecycle: scheduling → execution → heartbeats → XCom communication → retries/teardown
– Covers parsing & runtime contexts, isolation model, and security considerations
Architecture diagrams under task-sdk/docs/img/
closes: #49627
related: #51590
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named
{pr_number}.significant.rstor{issue_number}.significant.rst, in airflow-core/newsfragments.