Preparing your workspaceloading the brief, studio & editor…

Loadingone moment…

extracttransformloadorchestrate

Scenario

8.Categoryrevenuethatstaysput

Shoply sells online in three countries. Products sit in a browsable category tree — Electronics → Laptops → Gaming Laptops — and merchandising reorganises and renames it every few months. The store’s catalogue keeps no history: it shows you the tree as it is right now, and that is all it can do. The analytics team has two complaints — their dashboard takes forty seconds to load, and last week’s numbers changed after somebody renamed a category. Design the pipeline behind it.

This pipeline must answer

01Revenue per top-level category yesterday — from a table that reads in milliseconds, not forty seconds
02Merchandising renames a category today. Does last March's number change?
03Which categories existed in March, and what were they called then?

Guardrails

A reported number must not move afterwards
Assume order lines and the catalogue already exist upstream in the store’s database, and that order lines are only ever appended while the category tree is edited in place
Include a check before anything the dashboard reads, and a failure path
Avoid invalid cycles

Act 1 · Specify — answer all 8 and your spec flows0 / 8 answered

1Trigger & cadence

The dashboard is read each morning. What starts the run — and what else does that timing decide?

The deadline, and the thing about a nightly run that is easy to miss here.

This choice changes which questions you are asked next — and the pipeline you end up drawing.

◆ What you are committing to on the canvas

What is the last safe moment for yesterday's numbers to be ready?
e.g.
The team opens the dashboard first thing, so finished well before — with enough slack to re-run once without anybody noticing.
- configTrigger `Runs` = daily / hourly — leaving it unset is what fails the freshness check
What does the moment this runs decide, other than freshness?
e.g.
It also fixes when you look at the category tree. Anything merchandising changed and changed back between two runs never happened, as far as this pipeline is concerned.
- nodeOrchestration & contracts → Scheduler trigger for a clock; Amazon MWAA, Step Functions, Dagster or Prefect when one step has to wait for another to finish

New to this?Batch vs Streaming vs Micro-batch

Edit your answer here✓ captured

Markdown: **bold** · *italic* · # heading · - bullet

Act 2 · Architect

Practice studio

Ctrl/Cmd + C copyCtrl/Cmd + V pasteCtrl/Cmd + Z undoDelete remove

40%

Design review

The reviewer reads your pipeline.

Run the review to check your architecture against this scenario — a valid DAG, a trigger for the freshness SLA, a transform stage, a quality gate before publish, and a failure path — then defend it in a short interview.

Desktop only

This scenario runs a full workspace — editor, canvas and results side by side. It needs a laptop or desktop to be usable. Open this page on a bigger screen to start building.

Read the library instead Back to modules

Preparing your workspaceloading the brief, studio & editor…

extracttransformloadorchestrate

Scenario

8.Categoryrevenuethatstaysput

This pipeline must answer

01Revenue per top-level category yesterday — from a table that reads in milliseconds, not forty seconds
02Merchandising renames a category today. Does last March's number change?
03Which categories existed in March, and what were they called then?

Guardrails

A reported number must not move afterwards
Assume order lines and the catalogue already exist upstream in the store’s database, and that order lines are only ever appended while the category tree is edited in place
Include a check before anything the dashboard reads, and a failure path
Avoid invalid cycles

Act 1 · Specify — answer all 8 and your spec flows0 / 8 answered

1Trigger & cadence

The dashboard is read each morning. What starts the run — and what else does that timing decide?

The deadline, and the thing about a nightly run that is easy to miss here.

This choice changes which questions you are asked next — and the pipeline you end up drawing.

◆ What you are committing to on the canvas

What is the last safe moment for yesterday's numbers to be ready?
e.g.
The team opens the dashboard first thing, so finished well before — with enough slack to re-run once without anybody noticing.
- configTrigger `Runs` = daily / hourly — leaving it unset is what fails the freshness check
What does the moment this runs decide, other than freshness?
e.g.
It also fixes when you look at the category tree. Anything merchandising changed and changed back between two runs never happened, as far as this pipeline is concerned.
- nodeOrchestration & contracts → Scheduler trigger for a clock; Amazon MWAA, Step Functions, Dagster or Prefect when one step has to wait for another to finish

New to this?Batch vs Streaming vs Micro-batch

Edit your answer here✓ captured

Markdown: **bold** · *italic* · # heading · - bullet

Act 2 · Architect

Practice studio

Ctrl/Cmd + C copyCtrl/Cmd + V pasteCtrl/Cmd + Z undoDelete remove

40%

Design review

The reviewer reads your pipeline.

Desktop only

This scenario runs a full workspace — editor, canvas and results side by side. It needs a laptop or desktop to be usable. Open this page on a bigger screen to start building.

Read the library instead Back to modules

Category revenue that stays put · Pipeline Builder scenario · D8LooP