Delta Lake

When to Use

Load this skill when the user needs to:

• Create and manage Delta tables (DDL, partitioning, table properties)
• Write DML: INSERT, UPDATE, DELETE, MERGE (upsert, SCD2, CDC)
• Compact files (OPTIMIZE), apply data skipping (Z-ORDER BY)
• Vacuum stale files, time travel, restore snapshots
• Evolve schemas without breaking existing readers
• Read Delta tables in streaming or batch mode

SparkSession Setup

from pyspark.sql import SparkSession