Data Lake Platform

Build and operate production data lakes and lakehouses: ingest, transform, store in open formats, and serve analytics reliably.

When to Use

• Design data lake/lakehouse architecture
• Set up ingestion pipelines (batch, incremental, CDC)
• Build SQL transformation layers (SQLMesh, dbt)
• Choose table formats and catalogs (Iceberg, Delta, Hudi)
• Deploy query/serving engines (Trino, ClickHouse, DuckDB)
• Implement streaming pipelines (Kafka, Flink)
• Set up orchestration (Dagster, Airflow, Prefect)
• Add governance, lineage, data quality, and cost controls

Triage Questions