common-schema-audit

Installation
SKILL.md

Common-schema consumer drift audit

schema.Common (from github.com/redpanda-data/benthos/v4/public/schema) is the canonical type metadata that flows through meta(schema) between Avro / Parquet / CDC sources and downstream sinks. Every consumer of this metadata must:

  1. Handle every variant of schema.CommonType — or fail loudly with a useful error that names the missing case, not a generic "unsupported".
  2. Coerce values when the Go type of the message body doesn't match the schema-declared type — specifically the temporal-to-numeric and numeric-to-temporal bridges that the iceberg shredder implements via coerceTemporalToNumeric and the metadata-aware path in internal/impl/iceberg/shredder/temporal.go:208.

This skill produces a per-consumer report so reviewers can catch drift before it ships.

Why this matters

The "GF iceberg issue" was a value-vs-metadata mismatch class. Fixes closed the gap in each consumer:

  • iceberg output → temporal coerce + numeric metadata-aware scaling
  • parquet_encode → type coverage for Date/TimeOfDay/UUID/Map, temporal coerce bridges
  • confluent decoder / metadata parser → field-level logicalType, Debezium connect.name, duration
  • confluent JSON-Schema encoder → Date/TimeOfDay/UUID

A new consumer of schema.Common, or a new schema.CommonType variant added upstream in benthos, can re-introduce the same bug class without anyone noticing until a customer pipeline breaks. The audit catches the drift mechanically.

Installs
1
GitHub Stars
8.7K
First Seen
Jul 30, 2026
common-schema-audit — redpanda-data/connect