DATA ARCHITECTUREGovernance

Data Contracts, Schema Evolution & Governance

How data engineers design data that lasts — pick a topic on the left and its full breakdown loads here: the mental model, ERDs and worked schemas, trade-offs, edge cases, and the decisions that separate a durable model from a fragile one.

⏱ 18 min readTopics chapter readerLevel · Specialized & Applied
01 · Orientation

What You'll Master Here

a producer’s explicit promise to consumers, and schema evolution is how you change that API without breaking callers.

⏱ 4 min · Topic 1 of 8

Everything downstream — models, marts, features, dashboards — assumes upstream data keeps behaving the way it did yesterday. Nothing enforces that assumption. This chapter makes those dependencies explicit and safe to change: contracts, schema evolution, governance.

Start with the matrix below. Corvus Games ships one event stream to four consumers, and three things go wrong with it: a field is renamed, a field quietly changes meaning, and rows stop arriving. Four levels of defence, and no single one catches all three — which is exactly why this chapter has the topics it has.

Core mental model

A data contract is an API for data: a producer’s explicit promise to consumers, and schema evolution is how you change that API without breaking callers.

Why it matters

Undeclared data dependencies are the silent killer of data platforms: an upstream change breaks downstream with no warning. Contracts and disciplined evolution turn fragile implicit dependencies into explicit, safe ones.

data contract
A producer’s explicit promise about schema, semantics, quality, and ownership.
schema evolution
Changing a schema over time without breaking existing consumers.
compatibility
Whether a schema change still works for old/new readers (backward/forward/full).
lineage
The traced flow of data from sources through transformations to outputs.
Three upstream changes, four levels of defenceCorvus Games ships play.session_ended to four consumers. Pick a change, then pick the cheapest defence you think catches it before it reaches production.
Client build 4.31 renames session_seconds to duration_s in the event schema.Same value, same type, new name. The client team thought the old name was ambiguous.
Nothing selected — here is the shape of the answerEach of the three changes is stopped by a different layer, and no layer stops all three. Column tests catch a rename late and a meaning change never; a compatibility check catches the rename at build time and is blind to the other two; only the contract has anything to say about meaning and completeness.
ChangeNothingWarehouse testsRegistry compat checkFull contract
A field is renamedmissedcaught after shipcaught before shipcaught before ship
A field changes meaningmissedmissedmissedcaught before ship
Rows quietly stop arrivingmissedmissedmissedcaught before ship
Common mistake

Treating upstream tables as stable just because they exist. A silent upstream rename or type change breaks downstream models with no error at the boundary.

Better habit

Make data dependencies explicit with contracts.

Evolve schemas in compatible ways, or version.

Enforce contracts and track lineage automatically.

The big idea

Treat data like an API. A contract makes the producer accountable and the consumer safe; schema evolution is API versioning for data.

How to study this chapter

Predict before you click. In the matrix above, choose the cheapest defence you believe stops each change before it ships — the two you get wrong are the two topics to read most carefully.

Remember this

Data contracts make producer-consumer dependencies explicit, and schema evolution lets data change without breaking consumers, treat data like a versioned API.