anomaly-detection
Installation
SKILL.md
Overview
Anomaly Detection is a schema-level monitoring capability that automatically assesses data quality by evaluating freshness (is the table up-to-date?) and completeness (did we get the expected number of rows?). It uses ML models built from historical patterns — no custom metric definitions needed.
API Status: Public Preview (POST /api/data-quality/v1/monitors)
SDK Module: databricks.sdk.service.dataquality
When to Use This Skill vs. Lakehouse Monitoring
| Question | Use This Skill | Use Lakehouse Monitoring |
|---|---|---|
| "Did my pipeline break?" | Yes — detects stale/incomplete tables | No |
| "Is my revenue trending correctly?" | No | Yes — custom AGGREGATE/DERIVED/DRIFT metrics |
| "Which tables are unhealthy?" | Yes — schema-wide scan | No (per-table only) |
| "What's the average transaction value?" | No | Yes — custom business KPIs |
| "How quickly can I set up monitoring?" | Yes — minutes (enable on schema) | 2+ hours (custom metrics per table) |
| "Do I need ML model monitoring?" | No | Yes — InferenceLogConfig support |