Data Drift Strategy

Purpose: Detect and manage changes in input data distributions that degrade GenAI agent quality and ML pipeline reliability. Covers LLM input monitoring, embedding drift, RAG retrieval degradation, and classical feature distribution tracking.

When to Use This Skill

• Monitoring LLM input patterns in production (query length, topic distribution, language mix)
• Detecting embedding drift (vector space distribution shifts in RAG systems)
• Tracking RAG retrieval quality degradation (relevance scores, retrieval hit rates)
• Identifying user intent drift (new topics, out-of-scope queries, adversarial inputs)
• Monitoring input feature distributions in production ML systems
• Building data quality validation gates in ETL/ML pipelines
• Detecting schema drift (new columns, type changes, missing fields)
• Designing retraining triggers based on data distribution shifts
• Establishing data governance policies for model training data

Prerequisites