AWS Certified Machine Learning – Specialty — Question 30

A Machine Learning Specialist is building a logistic regression model that will predict whether or not a person will order a pizza. The Specialist is trying to build the optimal model with an ideal classification threshold.
What model evaluation technique should the Specialist use to understand how different classification thresholds will impact the model's performance?

Answer options

• A. Receiver operating characteristic (ROC) curve
• B. Misclassification rate
• C. Root Mean Square Error (RMSE)
• D. L1 norm

Correct answer: A

Explanation

The Receiver Operating Characteristic (ROC) curve is specifically designed to evaluate the performance of classification models at various threshold levels, making it the ideal choice for this scenario. The Misclassification rate provides a single measure of error but does not reflect the impact of different thresholds. Root Mean Square Error (RMSE) is more relevant for regression tasks, while L1 norm is used for regularization in model training, not for evaluating thresholds.