Databricks Certified Machine Learning Professional — Question 86

A Machine Learning Engineer wants to monitor the quality and stability of their machine learning model’s predictions over time. They have a Delta table, retail_inference_log, which records each model prediction along with input features, a timestamp, and (when available) the true label. They need to detect data drift and monitor model performance trends using Databricks Lakehouse Monitoring, ensuring that alerts are triggered if the distribution of predictions or input features changes significantly.

Which approach will set up monitoring for this use case?

Answer options

• A. Create a monitor with the Inference profile on the retail_inference_log table, specifying the timestamp column and the columns for model inputs, predictions, and labels. Configure the monitor to compute drift and performance metrics over time windows.
• B. Create a monitor with the Inference profile on the retail_inference_log table, and specify a recent batch of production data as the baseline table for drift detection. Use this recent production data to compare against new data for drift and performance monitoring.
• C. Create a monitor with the Snapshot profile on the retail_inference_log table, so that metrics are calculated over the entire table each time the monitor runs and therefore is able to compare new values with previous ones to compute data drift.
• D. Create a monitor with the Time Series profile on the retail_inference_log table, specifying the timestamp column and including model input, prediction columns and the true label column. This will track drift in features and predictions over time, and model performance could also be tracked using a custom metric.

Correct answer: A

Explanation

The correct answer is A because it sets up a monitor specifically geared towards calculating drift and performance metrics using defined time windows, which is crucial for monitoring changes over time. Options B, C, and D do not adequately address the requirement to compute metrics over time windows or do not use the Inference profile effectively for this particular use case.