Instruction: List and describe at least three key metrics that are crucial for monitoring the performance of a machine learning model in a production environment.
Context: This question evaluates the candidate's understanding of model monitoring essentials and their ability to identify and prioritize performance indicators critical to maintaining the health and accuracy of ML models post-deployment.
The way I'd explain it in an interview is this: I monitor three classes of metrics: system health, data health, and model behavior. System health includes latency, error rates, throughput, and resource usage. Data health includes missingness, schema drift, feature freshness, and distribution shifts. Model behavior includes accuracy or business KPIs, calibration, threshold metrics, and segment-level performance.
The exact mix depends on the use case, but the main principle is that a healthy endpoint is not the same as a healthy model. Production monitoring has to cover both.
What matters in an interview is not only knowing the definition, but being able to connect it back to how it changes modeling, evaluation, or deployment decisions in practice.
A weak answer lists latency and accuracy only, without covering data health, calibration, and segment-level behavior.