AWS Certified Machine Learning Engineer – Associate (MLA-C01) — Question 15

A company has deployed an XGBoost prediction model in production to predict if a customer is likely to cancel a subscription. The company uses Amazon SageMaker Model Monitor to detect deviations in the F1 score.
During a baseline analysis of model quality, the company recorded a threshold for the F1 score. After several months of no change, the model's F1 score decreases significantly.
What could be the reason for the reduced F1 score?

Answer options

• A. Concept drift occurred in the underlying customer data that was used for predictions.
• B. The model was not sufficiently complex to capture all the patterns in the original baseline data.
• C. The original baseline data had a data quality issue of missing values.
• D. Incorrect ground truth labels were provided to Model Monitor during the calculation of the baseline.

Correct answer: A

Explanation

The correct answer is A, as concept drift refers to changes in the data distribution over time, which can lead to a decline in model performance. Options B, C, and D do not directly address the issue of changes in the underlying data distribution affecting the model's predictive capabilities.