DBTScale & Expertise

Packages and Reusable dbt Code at Scale

How analytics engineers turn raw warehouse tables into trusted models — pick a topic on the left and its full breakdown loads here: the mental model, the compiled SQL dbt actually issues, live runs you can drive yourself, and the failure modes that quietly ship wrong numbers.

18 min readTopics chapter readerLevel · Hard
01 · Part 4 — Scale & Expertise begins here

Everything Hard About Packages, Named Up Front

given two candidate implementations, which one runs (resolution)? And given an install, what does it actually add to your graph and your warehouse spend (blast radius)?

4 min · Topic 1 of 10

Seventeen chapters in, Wavelength has written every macro it has ever called. This chapter changes that: a package splices a second — sometimes third — set of macro namespaces into your project’s resolution order, and occasionally ships nodes that build into your warehouse, on your bill. Two threads run through everything below: resolution (which implementation wins, and whether it stays the same implementation tomorrow) and blast radius (what runs, or builds, that you did not write). Section 2 asks resolution over time; Sections 5 and 6 ask it over space; Section 1 prices blast radius directly, and Section 4 prices it for a refactor specifically.

This is a run-it-yourself chapter — seven widgets, one per section, each a real dbt deps, a real codegen command, or a real resolution trace against Wavelength’s actual packages.yml. Start with the deps run below: watch what it changes on disk, and what it deliberately does not change in the graph.

dbt deps — packages.yml already carries dbt_utils (Ch 8) and dbt_expectations (Ch 7); this run adds audit_helper and codegen
dbt deps
  1. $ dbt deps
  2. Installing dbt-labs/dbt_utils
  3. Installed from version 1.4.1
  4. Up to date!
  5. Installing metaplane/dbt_expectations
  6. Installed from version 0.10.10
  7. Up to date!
  8. Installing dbt-labs/audit_helper
  9. Installed from version 0.14.0
  10. Installing dbt-labs/codegen
  11. Installed from version 0.14.1
  12. Installed 4 packages:
  13. - dbt-labs/dbt_utils@1.4.1
  14. - metaplane/dbt_expectations@0.10.10
  15. - dbt-labs/audit_helper@0.14.0
  16. - dbt-labs/codegen@0.14.1
new macros available11illustrative, not hand-counted — a delta, never a project-wide total; see the callout below
new files in dbt_packages/46illustrative install footprint, not hand-counted
graph nodes00 → 0. This is the lesson, not a footnote.
dbt-core #10760, OPEN — a green dbt parse over code that cannot build
Core mental model

A package is a second set of macro namespaces spliced into your project’s resolution order, plus — sometimes — nodes that build on your bill. Every hard question in this chapter reduces to one of two follow-ups: given two candidate implementations, which one runs (resolution)? And given an install, what does it actually add to your graph and your warehouse spend (blast radius)?

Why it matters

A package is the first dependency in this course that is neither your own code nor a warehouse feature — it is a third party’s SQL and Jinja, running inside your project, resolved by rules most engineers have never had to read closely because most days the defaults just work. The day they stop working — a silently shadowed macro, a silently ignored schema override, a silently expensive install — is the day this chapter’s two threads stop being theory. Interviewers ask about this because it is where “I know dbt” and “I have shipped dbt at scale” actually diverge.

misconception #1, previewed
Installing a package is additive and safe. It is not: a package can shadow a macro you already call, be silently ignored if it ships generate_schema_name, or add real graph nodes to your bill at the next parse — Section 1 shows all three, and none of them raises an error.
misconception #2, previewed
audit_helper proves a refactor is correct. It proves the two result sets are set-identical over the columns you compared — nothing about which side is right, and nothing about the columns you excluded. Section 4, the chapter’s flagship, walks this apart row by row.
resolution vs blast radius
Resolution is “which implementation runs” — Sections 2, 5 and 6. Blast radius is “what runs, or builds, that you did not write” — Sections 1 and 4. Most package incidents are one of these two questions answered wrong, silently.
This chapter, section by section
sectionthe one big ideawidget
1 — What dbt deps Actually Diddeps writes files; the graph moves at the next parse, and only if there is something to parsedbt-deps-delta
2 — Declaring a Dependencyhub/git/local answer “same bytes tomorrow?” differently; a lock file is what makes a range a factdbt-dependency-clock ⭐
3 — Four Packages, Four Questionseach package answers a different question; codegen removes typing, never judgmentdbt-codegen-bench
4 — Proving the Refactor Row-for-Rowaudit_helper proves set-equivalence, never correctness — and the denominator is a choicedbt-refactor-audit ⭐
5 — Building the Wavelength Packageship what takes relations and columns as arguments; keep what decides what to builddbt-package-extraction
6 — Whose Macro Winsa custom search_order replaces the default list entirely — an omitted root project is never even trieddbt-macro-precedence
7 — When a Macro Should Not Existfour gates, all-or-nothing: bounded shape, visible arguments, one hop, grep-survivabilitydbt-macro-verdict

Two sections are marked ⭐ — the ones the review gates weight heaviest, because each one has an interactive commit-then-reveal moment that cannot be skimmed.

Common mistake

Skimming this chapter for the packages.yml syntax and skipping the widgets. The syntax is the least of it. Every failure this chapter cares about is silent by construction — a shadowed macro, an ignored schema override, an omitted root project in a search_order, a package that quietly ships models on your bill. None of those show up by reading YAML; they only show up by tracing resolution and blast radius through a real example, which is exactly what the seven widgets are built to force.

Better habit

Before trusting any package behaviour, ask explicitly which of the two threads it touches — resolution or blast radius — and answer that question out loud, not just “it works”.

Treat `dbt deps` output as a files-on-disk event and the next parse as the real graph event — never conflate the two when reasoning about what a package install actually did.

When a package result looks plausible, be more suspicious, not less — every failure mode in this chapter produces a plausible-looking, silently wrong answer.

Resolution and blast radius — keep both words in your head

Every section from here on is one of these two questions in a different costume. If a claim in this chapter feels surprising, ask which of the two it is answering — it is always one of them.

How to use the seven widgets

Predict before you reveal wherever the widget asks you to — the two ⭐ sections gate their reveal behind a commitment on purpose. Getting a prediction wrong costs nothing here and is the whole point; getting it wrong in production is the failure mode this chapter exists to prevent.

Remember this

A package is a second set of macro namespaces spliced into your resolution order, plus sometimes nodes on your bill — every hard question below is resolution, blast radius, or both.