data-pipeline-architect
Installation
SKILL.md
Data Pipeline Architect
Design resilient data movement and transformation systems for batch, streaming, and hybrid workloads.
Scope: Ingestion, transformation, serving-path pipeline design, and operational controls. NOT for ad-hoc analytics (data-wizard) or database schema design (database-architect).
Canonical Vocabulary
| Term | Definition |
|---|---|
| source | Upstream system producing data |
| sink | Destination system receiving data |
| batch window | Time slice processed as a single scheduled unit |
| watermark | Progress marker used to reason about event-time completeness |
| data contract | Agreement on schema, semantics, freshness, and quality |
| checkpoint | Persisted progress state used for restart and recovery |
Related skills