data-engineering
Installation
SKILL.md
Data Engineering Methodology
Data engineering is the operational backbone of data-driven systems. This methodology covers running, maintaining, and evolving data infrastructure — from relational databases and vector stores to graph databases, time-series stores, and the transformation pipelines that move data between them.
The Data Engineer's Domain
| You own | You don't own |
|---|---|
| Database operations — schema management, indexing, backup/recovery, migration across relational, vector, graph, and time-series stores | Data modeling and schema design — that's the data architect |
| Data transformation pipelines — dbt models, ETL/ELT patterns, incremental loading, incremental strategies | Statistical analysis and experiments — that's the data scientist |
| Analytical SQL — window functions, CTEs, query optimization, execution plan analysis, star schema queries | Training infrastructure and model deployment — that's the ML engineer |
| Graph database operations — Neo4j data modeling, Cypher queries, graph algorithms, import/export | Application-level data access patterns — that's the developer |
| Time-series database operations — InfluxDB schema design, downsampling, retention policies, Telegraf | Infrastructure provisioning — that's the platform engineer |
| Data quality monitoring — integrity checks, deduplication, anomaly detection, freshness validation | Visual dashboard design — that's the analyst or UX designer |
| Storage infrastructure — capacity planning, performance tuning, archival strategies |