Data CDC Patterns

Purpose

Design reliable change data capture pipelines that stream database changes to downstream systems with minimal latency, exactly-once semantics, and schema evolution handling. Covers Debezium, Kafka Connect, AWS DMS, and custom CDC patterns.

Agent Protocol

Trigger

Exact user phrases: "CDC", "change data capture", "Debezium", "Kafka Connect", "AWS DMS", "log-based CDC", "trigger-based CDC", "timestamp-based CDC", "database replication", "incremental streaming", "capture", "change stream", "binlog", "WAL", "write-ahead log", "oplog", "version 2.0", "Debezium connector".

Input Context

Before activating, verify:

• Source database type (MySQL, Postgres, MongoDB, Oracle, SQL Server, DB2)
• Target sink (Kafka topic, data lake, warehouse, search index)
• Volume (transactions per second, rows changed per hour)
• Source database config (binlog format, WAL level, archive logs)
• Schema evolution frequency
• Consumer requirements (ordering, partitioning, exactly-once)
• Existing infrastructure (Kafka cluster, Kafka Connect, K8s)