APACHE AIRFLOWFoundations

Writing Your First DAG & Reading the UI

How data engineers make pipelines run themselves — pick a topic on the left and its full breakdown loads here: the mental model, runnable DAGs, the failure modes that wake people at 3am, and the judgment that separates a scheduled script from an orchestrated system.

⏱ 18 min readTopics chapter readerLevel · Easy
01 · Orientation

What You’ll Master Here

The DAG file is what you write; the UI is what Airflow made of it. Learn to look at a screen and name the object and state you are seeing.

⏱ 4 min · Topic 1 of 8

Three chapters of groundwork earn you this one. You are going to write a complete, working DAG, and then read the screens it produces — because the Airflow UI is not a dashboard, it is a direct view of the objects from Chapter 3 in the states from Chapter 3.

The second half matters more than it looks. Most people learn the UI by clicking things during an incident, which is the worst possible time to discover that Clear does not do what they assumed.

Core mental model

The DAG file is what you write; the UI is what Airflow made of it. Learn to look at a screen and name the object and state you are seeing.

Why it matters

A DAG you can write from memory is the unit of everything that follows. Every later chapter — scheduling, XComs, branching, dynamic mapping — is a modification to the shape you build here. And every later debugging chapter assumes you can read a Grid view without hesitating.

@dag
The decorator that turns a function into a DAG definition. The function body builds the graph; calling it registers the DAG.
@task
The decorator that turns a function into a task. Calling one inside the DAG function creates a node and wires dependencies.
Grid view
The primary interface for inspecting DAG runs and task states — a matrix with tasks as rows and runs as columns.
Clearing
Resetting a task instance so it runs again. It increments try_number, sets max_tries to 0, and sets the state to none.
Build a working DAG, one decision at a time
from datetime import datetime

from airflow.sdk import dag, task
Nothing Airflow-specific yet. These are the only things that belong at module level besides the DAG itself — Chapter 3 explains why that matters.
Step 1 of 5
Common mistake

Learning the UI buttons for the first time during a production incident. You discover under pressure that Clear only reset one task and the rest of the run is still stuck, or that Trigger created a second run instead of fixing the first. Both cost more time than the original failure.

Better habit

Build the DAG in the order shown — DAG first, then tasks, then dependencies. It matches how you will debug it.

Before using a UI action on production, know what it does to state. The simulator later in this chapter is the cheap way to find out.

Name what you are looking at. “A task instance in upstream_failed” is a better thought than “a red square”.

The UI is a state table

Every coloured square is one task instance in one state. Once you see the UI that way rather than as a dashboard, reading it becomes mechanical.

Remember this

Write the DAG, then learn to read what Airflow made of it. The UI is not decoration — it is the state machine, rendered.