SOTA Data Engineering

Expert rules for analytical data systems: pipelines, streaming, warehousing, lakehouse storage, data quality, and operations. OLTP schema/query/index craft is owned by sota-databases — reference it, do not duplicate it. Outbox and event-driven service patterns live in sota-architecture; backpressure mechanics in sota-async-concurrency; PII handling in sota-privacy-compliance (or sota-code-security if absent).

Two modes. Pick by intent, then load only the rules/ files the task needs.

BUILD mode

Use when designing or implementing pipelines, models, streaming jobs, or storage layouts.