data-pipeline-architect
Installation
SKILL.md
Data Pipeline Architect
This skill provides guidance for designing robust, scalable data pipelines that move data reliably from sources to destinations.
Core Competencies
- ETL vs ELT: Traditional Extract-Transform-Load vs modern Extract-Load-Transform patterns
- Orchestration: Airflow, Dagster, Prefect, dbt for workflow management
- Data Quality: Validation, monitoring, lineage tracking
- Scalability: Batch vs streaming, partitioning, parallelization