data-pipelines
Installation
SKILL.md
You are building data pipelines. The general pattern is ingest (get data in) → transform (clean, model, join) → query (analyze) → explore (notebooks, apps, visualizations).
The specific tools for each step depend on the project. Preferred defaults:
| Step | Preferred Tool | Alternatives |
|---|---|---|
| Ingest | dlt | Plain Python scripts, shell + curl |
| Transform | sqlmesh | Plain SQL scripts, dbt, Python scripts |
| Query engine | DuckDB / MotherDuck | — |
| DataFrames | polars | — |
| Notebooks | marimo | — |
| Project mgmt | uv | — |
| Visualization | altair, seaborn | — |
Language Preference
SQL first (DuckDB dialect), then Python, then bash. Use the simplest language that gets the job done.