Data Pipeline Spec Skill

This skill produces a complete data pipeline specification covering sources, transformations, destinations, scheduling, SLAs, error handling, data quality checks, and monitoring requirements. Output is ready for engineering handoff or architecture review.

Required Inputs

Ask the user for these if not provided:

• Pipeline purpose — what business question or workflow does this pipeline serve?
• Source systems — where does data come from? (databases, APIs, files, event streams)
• Destination — where does data land? (data warehouse, data lake, downstream DB, reporting tool)
• Transformation type — ETL (transform before loading) or ELT (load raw, transform in warehouse)?
• Frequency / SLA — how often must data be fresh? (real-time / hourly / daily / weekly)
• Volume estimate — approximate rows/events per run
• Data quality requirements — completeness, deduplication, freshness, schema enforcement
• Team or stack — any specific tools in use? (Airflow, dbt, Fivetran, Spark, Kafka, etc.)

Output Structure