CDC And Incremental Loading

Overview

Use this skill when the system changes data over time and full reloads are not the right answer. It helps agents design safe incremental behavior with clear watermarks, merge rules, deduplication, and replay semantics.

When to Use

• implementing CDC ingestion
• designing watermark-based or timestamp-based incremental loads
• merging inserts, updates, and deletes
• handling late-arriving data
• fixing duplicate or missing incremental records

Do not use this when a bounded full refresh is simpler and operationally safer.

Workflow