After deploying an ML model to production — models decay silently without monitoring.
You are a senior {{role}} brought in to help a developer or tech professional complete a {{use_case}} task. # Context - Pack: Developers & Tech Professionals - Category: Machine Learning & AI Engineering - Use case: ML Model Monitoring System - Source task: - Design a model monitoring system for {{describe_the_ml_model_in_production}}. - Step 1: Performance Monitoring: define operational metrics (latency, throughput, error rate) and model quality metrics (if ground truth is available with delay). - Step 2: Data Drift Detection: choose a statistical test (PSI, KS test, Jensen-Shannon divergence) for detecting input feature drift, and implement it. - Step 3: Prediction Drift: monitor output distribution changes over time. - Step 4: Alerting: define thresholds and alerting rules. - Step 5: Retraining Triggers: when should drift automatically trigger model retraining vs. manual investigation? # Goal Operational and quality metrics, drift detection implementation, alerting thresholds, and a retraining trigger decision framework. # Constraints - Treat this as a sequential workflow where each step builds on the previous step. - Keep every step clearly labeled and easy to run separately if needed. - Avoid generic filler, vague advice, and unsupported claims. - Make the output specific, practical, and ready to use. # Output Operational and quality metrics, drift detection implementation, alerting thresholds, and a retraining trigger decision framework.
{{double-curly}} with your real context.After deploying an ML model to production — models decay silently without monitoring.
Data drift is the most common reason a well-performing model degrades in production — monitor input distributions as carefully as prediction accuracy.
Debug this problem systematically. Identify the root cause, explain why it is happening, provide the fix, and explain how to prevent it in future.
Design the high-level architecture for this system. Cover components, data flow, scaling strategy, and key design decisions.
Recommend the best no-code or low-code tool stack for the stated goal, with implementation guidance.
Design the complete analysis approach for the stated question. Include the analytical method, the steps to execute it, and the format for presenting findings.