monitor-model-drift

Detect and alert on data drift and concept drift in production ML models using statistical tests and automated monitoring.

Expected: Configuration file created with thresholds matching your model's tolerance.

On failure: Start with conservative thresholds (PSI > 0.2, KS p-value < 0.01) and tune based on false positive rate.

Implement comprehensive model drift monitoring using Evidently AI, statistical tests (PSI, KS), and custom metrics to detect data drift and concept drift in production ML systems. Set up automated alerting and reporting workflows to catch degradation before it impacts business metrics. Use when production models show unexplained performance degradation, when new data distributions differ from training data, when seasonal shifts affect input features, or when regulatory requirements mandate model monitoring. Source: pjt222/development-guides.