Preparing your workspaceloading the brief, studio & editor…

Loadingone moment…

extracttransformloadorchestrate

Scenario

3.Multi-sourcefan-inorchestration

Coordinate several independent sources (orders, payments, clickstream) into one downstream mart: fan them in, join on a shared key, and fail safely if a source is late or missing rather than publishing a half-built mart.

This pipeline must answer

01Are all the sources present and complete before we build the mart?
02How are orders, payments, and events joined into one correct row?
03If one source is late or missing, do we publish a half-built mart — or wait?

Guardrails

At least two sources joined on a key
Wait for all sources before publishing; alert on a late/missing source
Idempotent mart write
Avoid invalid cycles

Act 1 · Specify — answer all 8 and your spec flows0 / 8 answered

1Trigger & cadence

Three feeds, one mart, one deadline. What starts the run?

The mart is only as fresh as its slowest input — design the trigger around that, not around a hopeful clock time.

This choice changes which questions you are asked next — and the pipeline you end up drawing.

◆ What you are committing to on the canvas

When is the mart actually read, and by whom?
e.g.
If it backs a morning dashboard, finished by 06:00 is the deadline; the cadence follows from that.
- configTrigger `Runs` = daily / hourly — leaving it unset is what fails the freshness check
Does the clock start it, or the last feed landing?
e.g.
A cron at 02:00 fires whether or not payments arrived. A sensor on all three waits for the one that is late.
- nodeOrchestration & contracts → Scheduler trigger for a clock; Amazon MWAA, Step Functions, Dagster or Prefect when one step has to wait for another to finish

New to this?Batch vs Streaming vs Micro-batch

Edit your answer here✓ captured

Markdown: **bold** · *italic* · # heading · - bullet

Act 2 · Architect

Practice studio

Ctrl/Cmd + C copyCtrl/Cmd + V pasteCtrl/Cmd + Z undoDelete remove

40%

Design review

The reviewer reads your pipeline.

Run the review to check your architecture against this scenario — a valid DAG, a trigger for the freshness SLA, a transform stage, a quality gate before publish, and a failure path — then defend it in a short interview.

Desktop only

This scenario runs a full workspace — editor, canvas and results side by side. It needs a laptop or desktop to be usable. Open this page on a bigger screen to start building.

Read the library instead Back to modules

Preparing your workspaceloading the brief, studio & editor…

extracttransformloadorchestrate

Scenario

3.Multi-sourcefan-inorchestration

This pipeline must answer

01Are all the sources present and complete before we build the mart?
02How are orders, payments, and events joined into one correct row?
03If one source is late or missing, do we publish a half-built mart — or wait?

Guardrails

At least two sources joined on a key
Wait for all sources before publishing; alert on a late/missing source
Idempotent mart write
Avoid invalid cycles

Act 1 · Specify — answer all 8 and your spec flows0 / 8 answered

1Trigger & cadence

Three feeds, one mart, one deadline. What starts the run?

The mart is only as fresh as its slowest input — design the trigger around that, not around a hopeful clock time.

This choice changes which questions you are asked next — and the pipeline you end up drawing.

◆ What you are committing to on the canvas

When is the mart actually read, and by whom?
e.g.
If it backs a morning dashboard, finished by 06:00 is the deadline; the cadence follows from that.
- configTrigger `Runs` = daily / hourly — leaving it unset is what fails the freshness check
Does the clock start it, or the last feed landing?
e.g.
A cron at 02:00 fires whether or not payments arrived. A sensor on all three waits for the one that is late.
- nodeOrchestration & contracts → Scheduler trigger for a clock; Amazon MWAA, Step Functions, Dagster or Prefect when one step has to wait for another to finish

New to this?Batch vs Streaming vs Micro-batch

Edit your answer here✓ captured

Markdown: **bold** · *italic* · # heading · - bullet

Act 2 · Architect

Practice studio

Ctrl/Cmd + C copyCtrl/Cmd + V pasteCtrl/Cmd + Z undoDelete remove

40%

Design review

The reviewer reads your pipeline.

Desktop only

This scenario runs a full workspace — editor, canvas and results side by side. It needs a laptop or desktop to be usable. Open this page on a bigger screen to start building.

Read the library instead Back to modules

Multi-source fan-in orchestration · Pipeline Builder scenario · D8LooP