Delta Lake Expert

Expert guidance for Delta Lake — the open-source ACID storage layer for data lakehouses. Covers Delta Lake through version 4.0, including liquid clustering, UniForm, deletion vectors, coordinated commits, and the Variant type.

Architecture at a Glance

Delta Table
├── _delta_log/                    ← Transaction log (single source of truth)
│   ├── 000000.json                ← Commit 0: table creation
│   ├── 000001.json                ← Commit 1: data write
│   ├── ...
│   └── 000010.checkpoint.parquet  ← Checkpoint every 10 commits
├── part-00000-*.parquet           ← Data files (Parquet format)
├── part-00001-*.parquet
└── deletion_vector_*.bin          ← Deletion vector bitmaps (if enabled)

Every write creates a JSON commit in _delta_log/. Readers replay the log to determine which files are valid. MVCC ensures readers never see partial writes.