Airflow And Workflow Orchestration

Overview

Use this skill when the problem is orchestration rather than transformation logic. It helps agents design Airflow, MWAA, Cloud Composer, Azure Data Factory, Step Functions, Google Cloud Workflows, or Databricks Workflows patterns with explicit dependencies, retries, ownership, backfills, and publish-safe cutover behavior.

When to Use

• building or modifying Airflow DAGs
• choosing between scheduler-driven, event-driven, or platform-native orchestration
• designing scheduling, sensors, task dependencies, or retry policy
• coordinating ingestion, transformation, quality checks, and publish steps
• changing backfill, catchup, or SLA behavior

Do not use this as a substitute for the underlying processing skill. Orchestration coordinates work; it does not define the compute logic itself.

Workflow