SQL Market Data

objective

Execute sql market data work with reproducible research, explicit controls, and deployable outputs.

workflow

1. define source contracts, schema versions, and freshness objectives.
2. ingest data with replay support and deterministic normalization.
3. validate keys, timestamps, and point-in-time join behavior.
4. monitor quality metrics continuously and quarantine degraded feeds.
5. publish only when lineage, ownership, and quality thresholds are satisfied.

required diagnostics

• freshness, completeness, null-rate, and duplicate-rate trends.
• schema drift and breaking-change frequency across sources.
• point-in-time join integrity for features and labels.
• backfill and replay consistency versus canonical snapshots.
• predicate pushdown effectiveness and scan amplification
• point-in-time alignment for slowly changing dimensions