airflow
Airflow
Apache Airflow is the standard for data engineering pipelines. v3.0 (2025) introduces Event-driven Triggers and a modern React UI.
When to Use
- ETL/ELT: Scheduling nightly data warehouse loads.
- ML Ops: Retraining models when new data arrives.
- Dependency Management: "Run Task B only if Task A succeeds".
Core Concepts
DAGs (Directed Acyclic Graphs)
Defined in Python.
Task SDK
New in v3.0. Allows writing tasks in any language, not just Python.
More from g1joshi/agent-skills
template
Expert [skill-name] assistance covering [feature 1], [feature 2], and [feature 3]. Use when [working with X], [debugging Y], or [implementing Z].
34mariadb
MariaDB MySQL-compatible database with Galera clustering. Use for MySQL-compatible database needs.
6claude
Anthropic Claude AI models for analysis and coding. Use for AI assistants.
5javascript
JavaScript ES6+ programming including async/await, DOM manipulation, modules, and Node.js. Use for .js files and web development.
4typescript
TypeScript static typing with interfaces, generics, decorators, and type inference. Use for .ts files.
4python
Python programming with type hints, async/await, decorators, and package management. Use for .py files and data science.
4