DE Cost Optimization

When to Use

Load this skill when the user needs to:

Review a cloud bill and identify which pipelines, queries, or clusters drive cost
Right-size a Spark, Trino, ClickHouse, or Airflow cluster (instance type, memory, core count)
Reduce query scan costs via partition pruning, Z-order clustering, or file compaction
Move data to cheaper storage tiers (S3-IA, Glacier, ClickHouse cold volumes)
Implement spot/preemptible instances for Spark executors or Airflow workers
Decide whether to materialize a view or a summary table (refresh cost vs query savings)
Define or enforce a data lifecycle and retention policy
Set up AWS/GCP budget alerts, FinOps tagging, or per-team cost attribution
Run a FinOps review and produce a prioritized cost reduction plan