What You’ll Master Here
The template context is what Airflow knows about this run. XComs are what other tasks discovered during it. Both arrive too late for your file to have hard-coded them.
A DAG file is written once and runs thousands of times. Everything that differs between runs — which day, which partition, which config — has to arrive from somewhere at run time. There are exactly two channels: XComs carry values produced by other tasks, and the template context carries values Airflow knows about the run.
This chapter covers both properly: the full catalogue of variables you get for free, how those values differ between a scheduled run and a backfill, and how to push and pull XComs when the automatic version is not enough.
The template context is what Airflow knows about this run. XComs are what other tasks discovered during it. Both arrive too late for your file to have hard-coded them.
These two mechanisms are how a pipeline becomes parameterised by time. Get them right and one DAG file correctly processes any day in history. Get them wrong and your backfill quietly writes today’s data over last June — the failure that looks like success.
- Template context
- The dictionary of variables Jinja can reach — dates, the DAG and run objects, params, variables and connections.
- XCom
- Cross-communication. A small value one task publishes and another reads, stored as a row in the metadata database.
- template_fields
- The class attribute listing which of an operator’s arguments get rendered. Templating is opt-in, per argument.
Hard-coding a date in a DAG file “just for now”. The DAG is no longer a function of its run. Every backfill produces the same day, every re-run overwrites the same partition, and the bug is invisible because the code runs perfectly.
Derive every date from the template context, never from the wall clock.
Treat XCom as metadata transport, never as data transport.
Check template_fields before relying on Jinja in an argument, especially in operators you wrote.
Both exist because the file is static and the runs are not. Anything that varies per run comes from an upstream task or from the context — there is no third source.
XComs carry what tasks discovered; the context carries what Airflow knows. Together they make one file work for every day in history.