AWS Certified Machine Learning – Specialty — Question 245

A company’s data scientist has trained a new machine learning model that performs better on test data than the company’s existing model performs in the production environment. The data scientist wants to replace the existing model that runs on an Amazon SageMaker endpoint in the production environment. However, the company is concerned that the new model might not work well on the production environment data.

The data scientist needs to perform A/B testing in the production environment to evaluate whether the new model performs well on production environment data.

Which combination of steps must the data scientist take to perform the A/B testing? (Choose two.)

Answer options

• A. Create a new endpoint configuration that includes a production variant for each of the two models.
• B. Create a new endpoint configuration that includes two target variants that point to different endpoints.
• C. Deploy the new model to the existing endpoint.
• D. Update the existing endpoint to activate the new model.
• E. Update the existing endpoint to use the new endpoint configuration.

Correct answer: A, E

Explanation

The correct steps for A/B testing involve creating a new endpoint configuration with production variants for both models (Option A) and updating the existing endpoint to use this new configuration (Option E). Options B, C, and D do not adequately set up the necessary testing environment or involve the proper configurations for A/B testing in SageMaker.