schema-spec

Overview

Based on Fundamentals of Data Engineering (Reis & Housley) and Designing Data-Intensive Applications (Kleppmann). A schema spec is the data contract between producers and consumers. Kleppmann's rule: schema changes are the most dangerous category of change in a data system - the spec exists to make schema evolution explicit and backward-compatible.

The test: can a consumer engineer write a query or build an integration using only this document, without reading source code or asking the producer team?

Workflow

Step 1: Write the schema header