AWS Certified Machine Learning – Specialty — Question 218
A bank wants to use a machine learning (ML) model to predict if users will default on credit card payments. The training data consists of 30,000 labeled records and is evenly balanced between two categories. For the model, an ML specialist selects the Amazon SageMaker built-in XGBoost algorithm and configures a SageMaker automatic hyperparameter optimization job with the Bayesian method. The ML specialist uses the validation accuracy as the objective metric.
When the bank implements the solution with this model, the prediction accuracy is 75%. The bank has given the ML specialist 1 day to improve the model in production.
Which approach is the FASTEST way to improve the model's accuracy?
Answer options
- A. Run a SageMaker incremental training based on the best candidate from the current model's tuning job. Monitor the same metric that was used as the objective metric in the previous tuning, and look for improvements.
- B. Set the Area Under the ROC Curve (AUC) as the objective metric for a new SageMaker automatic hyperparameter tuning job. Use the same maximum training jobs parameter that was used in the previous tuning job.
- C. Run a SageMaker warm start hyperparameter tuning job based on the current model’s tuning job. Use the same objective metric that was used in the previous tuning.
- D. Set the F1 score as the objective metric for a new SageMaker automatic hyperparameter tuning job. Double the maximum training jobs parameter that was used in the previous tuning job.
Correct answer: C
Explanation
The fastest way to improve the model's accuracy is by running a SageMaker warm start hyperparameter tuning job, as indicated in option C. This approach allows the model to leverage the results from the previous tuning job while staying focused on the previously established objective metric. Options A and B suggest methods that may take longer, and option D changes the objective metric, potentially complicating the optimization process and causing delays.