Model Drift Management

Purpose: Detect model performance degradation in GenAI agents and traditional ML systems. Covers LLM output quality monitoring, prompt regression detection, model version change management, and classical statistical drift detection.

When to Use This Skill

• Monitoring LLM output quality in production (hallucination rate, format compliance, coherence)
• Detecting prompt regression after prompt edits or model version changes
• Managing model version transitions (provider silent updates, planned migrations)
• Implementing LLM-as-judge evaluation pipelines for continuous quality monitoring
• Monitoring model performance in production (accuracy decay, prediction shifts)
• Implementing drift detection pipelines (concept drift, prior probability shift)
• Designing change management workflows for model retraining and replacement
• Building model governance and versioning policies
• Setting up alerting thresholds for model degradation

Prerequisites