D8LooP - Practise Data Engineering

01 · Orientation

From an Empty Folder to a Table

A dbt project is a folder with a dbt_project.yml in it. Everything else — models, macros, tests — is convention that file points at. Your credentials deliberately live somewhere else entirely.

⏱ 4 min · Topic 1 of 8

Chapter 1 argued for dbt. This chapter builds something with it: an empty directory, one command, some configuration, and a table in the warehouse that dbt created. By the end you will have run five commands and be able to explain what every one of them did.

Start with the shape of what `dbt init` produces. Most of these files you will never touch; three of them you will touch constantly. Select each one below to see which is which.

Core mental model

A dbt project is a folder with a dbt_project.yml in it. Everything else — models, macros, tests — is convention that file points at. Your credentials deliberately live somewhere else entirely.

Why it matters

Setup is where most people lose their first day with dbt, and almost always to the same three things: a profile name that does not match, a YAML indentation error, or running from the wrong directory. None of those are conceptually hard, but each produces an error message that points somewhere unhelpful. Recognising them on sight is worth more than any amount of modelling theory in week one.

dbt_project.yml: The file whose presence defines a project. dbt walks up from your working directory looking for it, and gives up at the filesystem root. It names the project, names the profile to connect with, and sets folder-level defaults.
profiles.yml: Connection credentials. Lives outside the project by default, at ~/.dbt/profiles.yml, so it is never committed and can be reused across every project on the machine.
Adapter: The plugin that teaches dbt to speak one warehouse’s dialect — dbt-snowflake, dbt-bigquery, dbt-databricks, dbt-postgres, dbt-duckdb. Installed separately from dbt itself.
Target: A named set of connection settings within a profile — typically `dev` and `prod`. The same models build into different places depending on which target is active.

What `dbt init wavelength_analytics` actually creates — select a file

you edit this constantlydbt_project.ymlProject configuration. Its presence is what makes this folder a dbt project.Names the project, declares which profile to connect with, tells dbt where the model and macro folders are, and sets project-wide defaults such as materializations per folder. dbt walks up from your working directory looking for this file — no dbt_project.yml, no project.

The whole setup, start to finishworked example

Shell

$ pip install dbt-core dbt-duckdb          # engine + one adapter
$ dbt init wavelength_analytics             # scaffold the project

  Running with dbt=1.12.0
  Your new dbt project "wavelength_analytics" was created!

$ cd wavelength_analytics
$ dbt debug                                 # prove the connection works
  All checks passed!

$ dbt build                                 # build models, run their tests
  Completed successfully
  Done. PASS=3 WARN=0 ERROR=0 SKIP=0 TOTAL=3

Five commands. Everything else in this chapter is an explanation of what happened between them — and, more usefully, what to do when one of them does not print that.

Common mistake

Installing `dbt` from PyPI and expecting it to connect to your warehouse. dbt-core has no database drivers of its own. Without the matching adapter package you get an error about an unknown type at the first command that touches the connection — install `dbt-core` and `dbt-<your-warehouse>` together.

Better habit

Use DuckDB for your first project even if your company runs Snowflake. It removes credentials, networks and cost from the learning loop entirely, and every concept transfers unchanged.

Pin your dbt version in the project from day one. “It broke and nobody changed anything” is almost always a version that moved underneath a CI image.

Run `dbt debug` before you believe any other error message. Half of all early dbt problems are connection problems wearing a different costume.

The starter project has example models in it

`dbt init` leaves two example models in models/example/. Delete them once you have run them successfully — they exist to prove the setup works, and leaving them in place is the first small piece of clutter in a project that will accumulate plenty on its own.

Remember this

A folder with dbt_project.yml in it is a dbt project. Credentials live outside it, on purpose, and that single separation explains most of this chapter.

Your First Project: Setup, Profiles, and the Run Loop