Preparing your workspaceloading the brief, studio & editor…

Loadingone moment…

extracttransformloadorchestrate

Scenario

38.Thenumberamachineactson

A ride-hailing app prices each journey with a multiplier that reflects how busy a city block is right now. A pricing service asks for that number thousands of times a second and charges people whatever it is told. Design the pipeline behind it.

This pipeline must answer

01The multiplier for a city block is forty seconds old. Is that still safe to charge somebody?
02The pipeline stops. What does the pricing service do next, and who decided that?
03A rider is charged 2.4x and complains. Can you show what the pipeline believed at that second?

Guardrails

The consumer is a machine and there is no human between your number and a fare
Supply and demand arrive as two separate streams, and one of them is systematically stale in exactly the places that matter
A rider may dispute a fare weeks later
Include a check before anything is published, and a failure path
Avoid invalid cycles

Act 1 · Specify — answer all 8 and your spec flows0 / 8 answered

1Trigger & cadence

The pricing service asks for a multiplier thousands of times a second and expects an answer in single-digit milliseconds. What produces that number, and how often?

Two different clocks: how fast the answer is served, and how fresh the answer is.

This choice changes which questions you are asked next — and the pipeline you end up drawing.

◆ What you are committing to on the canvas

What is running, and what is it doing between requests?
e.g.
Say what computes the number and what the pricing service actually reads. If they are the same thing, say why that is safe at thousands of requests a second.
- configTrigger `Runs` = daily / hourly — leaving it unset is what fails the freshness check
How old is the number the service reads, at worst?
e.g.
Give a figure and say what it is made of. “Real time” is not a figure.
- nodeOrchestration & contracts → Scheduler trigger for a clock; Amazon MWAA, Step Functions, Dagster or Prefect when one step has to wait for another to finish

New to this?Batch vs Streaming vs Micro-batch

Edit your answer here✓ captured

Markdown: **bold** · *italic* · # heading · - bullet

Act 2 · Architect

Practice studio

Ctrl/Cmd + C copyCtrl/Cmd + V pasteCtrl/Cmd + Z undoDelete remove

40%

Design review

The reviewer reads your pipeline.

Run the review to check your architecture against this scenario — a valid DAG, a trigger for the freshness SLA, a transform stage, a quality gate before publish, and a failure path — then defend it in a short interview.

Desktop only

This scenario runs a full workspace — editor, canvas and results side by side. It needs a laptop or desktop to be usable. Open this page on a bigger screen to start building.

Read the library instead Back to modules

Preparing your workspaceloading the brief, studio & editor…

extracttransformloadorchestrate

Scenario

38.Thenumberamachineactson

This pipeline must answer

01The multiplier for a city block is forty seconds old. Is that still safe to charge somebody?
02The pipeline stops. What does the pricing service do next, and who decided that?
03A rider is charged 2.4x and complains. Can you show what the pipeline believed at that second?

Guardrails

The consumer is a machine and there is no human between your number and a fare
Supply and demand arrive as two separate streams, and one of them is systematically stale in exactly the places that matter
A rider may dispute a fare weeks later
Include a check before anything is published, and a failure path
Avoid invalid cycles

Act 1 · Specify — answer all 8 and your spec flows0 / 8 answered

1Trigger & cadence

The pricing service asks for a multiplier thousands of times a second and expects an answer in single-digit milliseconds. What produces that number, and how often?

Two different clocks: how fast the answer is served, and how fresh the answer is.

This choice changes which questions you are asked next — and the pipeline you end up drawing.

◆ What you are committing to on the canvas

What is running, and what is it doing between requests?
e.g.
Say what computes the number and what the pricing service actually reads. If they are the same thing, say why that is safe at thousands of requests a second.
- configTrigger `Runs` = daily / hourly — leaving it unset is what fails the freshness check
How old is the number the service reads, at worst?
e.g.
Give a figure and say what it is made of. “Real time” is not a figure.
- nodeOrchestration & contracts → Scheduler trigger for a clock; Amazon MWAA, Step Functions, Dagster or Prefect when one step has to wait for another to finish

New to this?Batch vs Streaming vs Micro-batch

Edit your answer here✓ captured

Markdown: **bold** · *italic* · # heading · - bullet

Act 2 · Architect

Practice studio

Ctrl/Cmd + C copyCtrl/Cmd + V pasteCtrl/Cmd + Z undoDelete remove

40%

Design review

The reviewer reads your pipeline.

Desktop only

This scenario runs a full workspace — editor, canvas and results side by side. It needs a laptop or desktop to be usable. Open this page on a bigger screen to start building.

Read the library instead Back to modules

The number a machine acts on · Pipeline Builder scenario · D8LooP