Data Observability

Use this skill when the problem is not "is the code deployed?" but "is the data still arriving, complete, and on time?"

Goal

Detect broken or degraded data delivery before downstream users notice.

Working approach

1. Identify the unit of delay or completeness:
   • batch run
   • partition
   • row count
   • stream offset
   • commit version
2. Define the consumer-facing SLA first.
3. Track both freshness and volume, not just task success.
4. Alert on states that require action, not on every anomaly.