common-schema-audit
Installation
SKILL.md
Common-schema consumer drift audit
schema.Common (from github.com/redpanda-data/benthos/v4/public/schema) is the canonical type metadata that flows through meta(schema) between Avro / Parquet / CDC sources and downstream sinks. Every consumer of this metadata must:
- Handle every variant of
schema.CommonType— or fail loudly with a useful error that names the missing case, not a generic "unsupported". - Coerce values when the Go type of the message body doesn't match the schema-declared type — specifically the temporal-to-numeric and numeric-to-temporal bridges that the iceberg shredder implements via
coerceTemporalToNumericand the metadata-aware path ininternal/impl/iceberg/shredder/temporal.go:208.
This skill produces a per-consumer report so reviewers can catch drift before it ships.
Why this matters
The "GF iceberg issue" was a value-vs-metadata mismatch class. Fixes closed the gap in each consumer:
icebergoutput → temporal coerce + numeric metadata-aware scalingparquet_encode→ type coverage for Date/TimeOfDay/UUID/Map, temporal coerce bridgesconfluentdecoder / metadata parser → field-level logicalType, Debezium connect.name, durationconfluentJSON-Schema encoder → Date/TimeOfDay/UUID
A new consumer of schema.Common, or a new schema.CommonType variant added upstream in benthos, can re-introduce the same bug class without anyone noticing until a customer pipeline breaks. The audit catches the drift mechanically.