One Command, One Bit of Truth
where does the run's state live? In the dbt process (and its target/ directory), in Airflow's metadata database, or in a vendor's. Nothing else about the four options actually differs.
An orchestrator does not run your dbt project. It starts a dbt process, waits for it, and stores one boolean. Everything hard about orchestrating dbt comes from that boolean being a lossy summary of the graph as Chapter 12 left it — 23 nodes, 15 of them buildable — and from dbt's own record of what happened, target/run_results.json, dying with the container that produced it.
Pick a recovery question in the widget below before you read anything else. One of the four is unanswerable by every setup on the list, and finding out which one is the fastest way to stop believing the thing most teams believe about task granularity.
Every orchestration decision in this chapter reduces to one question: where does the run's state live? In the dbt process (and its target/ directory), in Airflow's metadata database, or in a vendor's. Nothing else about the four options actually differs.
Chapter 1 said dbt is not an orchestrator; this chapter is the invoice for that sentence. Wavelength ships a nightly production build at 06:00 UTC, and on Monday 2026-07-27 it failed halfway. Twelve nodes committed to the warehouse and stayed committed — one of them still running when fct_listens errored — one errored, two were never attempted, and every downstream human saw a system that looked fine. The cost of a bad orchestration decision is never the failed run — you get told about that. It is the forty minutes at 08:00 spent working out which model broke, whether it is safe to re-run, and who to tell, on a Monday morning when weekly_listening_dashboard is on a screen in an exec meeting.
- dbt platform (formerly dbt Cloud)
- The hosted product. Renamed 28 May 2025, at dbt Labs' Launch Showcase and confirmed in dbt's 2025 release notes — Chapter 1 owns the rename and this chapter just uses it. The scheduler surface is branded Orchestrator. Tiers: Developer (free), Starter ($100/user/month), Enterprise, Enterprise+.
- Still spelled "dbt Cloud" in machine-facing surfaces
- The Airflow provider package is apache-airflow-providers-dbt-cloud, the namespace is airflow.providers.dbt.cloud, the API path is /dbt-cloud/api-v2, and the environment variables are DBT_CLOUD_ACCOUNT_ID, DBT_CLOUD_ENVIRONMENT_ID and DBT_CLOUD_SERVICE_TOKEN. Do not "fix" any of those.
- astronomer-cosmos
- The open-source library that renders a dbt project as Airflow tasks. Version 1.15.0, released 1 July 2026; requires Python 3.10+, declares apache-airflow>=2.9.0, and its CI covers Airflow 3.0 to 3.3. It now runs dbt Core or dbt Fusion projects.
- target/run_results.json
- Chapter 2's artifact: the per-node status and timing record of one invocation. It is what dbt retry reads, and it is the single file whose survival decides whether resumption is possible. It lives in target/, which lives on a filesystem, which may not outlive the process.
- A task is a failure domain
- The Airflow KB's idea, not this chapter's. What a task retries, what it skips, and what it isolates are all decided by where you drew its boundary. Everything in Section 2 is that idea applied to a dbt graph.
| What runs | Where the run state lives | Who operates it | |
|---|---|---|---|
| One Airflow task | One `dbt build`, one dbt process | In target/ on whatever filesystem the task got | You |
| Four Airflow tasks | Four `--select` invocations | Airflow, at layer resolution; target/ for the rest | You |
| Task per model (Cosmos) | 15 rendered tasks, 1 or 15 dbt processes | Airflow's metadata database, per node | You, plus a parse bill |
| dbt platform Orchestrator | One job, one or more steps | The vendor's, including target/ | They do |
Read the third column twice. It is the only one that changes anything, and the fourth is the one people actually decide on.
Saying "Airflow orchestrates my models". It does not. dbt schedules models; Airflow schedules invocations. `ref()` is the only thing that builds dbt's build order, and the moment somebody writes `stg_listens >> int_listens_enriched` in a Python file, the project has two dependency graphs that can disagree. Rename a model and dbt follows; the hand-written Airflow edge does not, and now the orchestrator is enforcing an order the project abandoned.
Choosing task-per-model because it will give you more observability. It gives you finer granularity of the boolean, which is a different thing. Fifteen green squares — three of them seeds and a snapshot, not models, and the count changes the day somebody adds a node — mean fifteen nodes built, not fifteen nodes correct. What actually tells you the data is right — tests, source freshness, run_results.json — is identical under all four setups, so you paid a parse bill and a manifest-publication step for a prettier failure map.
Before choosing a granularity, write down the recovery question you are actually trying to answer. If you cannot name one, one task is the answer.
Treat "where does target/ live between attempts?" as a design question you answer on day one, not a thing you discover during an incident.
When somebody proposes hand-writing Airflow edges between dbt models, ask what happens on the next rename. That question ends the conversation faster than an argument does.
Everything Section 2 does with granularity is that principle applied to a dbt graph: what a task retries is what you drew inside it, and what it can page about is what it can name. Read it there once; this chapter will not re-derive it, and will lean on it heavily.
Idempotency and safe task design — Airflow KBNo terminal output here was captured from a live dbt Core 1.12.0 run. Message strings reproduce documented or source text — dbt's Database Error wrapper line, the retry documentation, the Cosmos operator descriptions — and the timings are this KB's own published Wavelength figures: 39.80s for fct_subscription_revenue, 0.61s for a four-row seed, 0.21s for a staging view, 2.08s for the snapshot. Where a message belongs to the warehouse rather than to dbt, it is left as a placeholder rather than invented. Chapters 8 and 11 shipped the same note for the same reason.
dbt Core 1.12.0 (16 July 2026), Airflow 3.3.0 (6 July 2026), astronomer-cosmos 1.15.0 (1 July 2026), apache-airflow-providers-dbt-cloud 4.9.2 (22 June 2026, minimum Airflow 2.11.0). One sentence about the far future and then it is Chapter 22's: dbt Core v2.0 is at 2.0.0-alpha.5 as of 20 July 2026, Apache 2.0 licensed.
An orchestrator starts a dbt process and stores one boolean; every option in this chapter is a different answer to "where does the run's state live?", and none of them answers "is the data right?".