Data Pipeline Architect

Design for operability first. Fancy topology that nobody can run is bad architecture.

Scope

Use this skill for:

• batch, micro-batch, streaming, or hybrid topology decisions
• event ingestion, CDC, SaaS ingestion, and warehouse landing patterns
• orchestration and dependency design
• partitioning, ordering, idempotency, and replay strategy
• data contracts, schema evolution, and producer-consumer boundaries
• latency, throughput, cost, and reliability tradeoffs
• migration from fragile legacy ETL to a durable platform model

Use this skill when