AWS Certified Machine Learning – Specialty — Question 287

An insurance company developed a new experimental machine learning (ML) model to replace an existing model that is in production. The company must validate the quality of predictions from the new experimental model in a production environment before the company uses the new experimental model to serve general user requests.

New one model can serve user requests at a time. The company must measure the performance of the new experimental model without affecting the current live traffic.

Which solution will meet these requirements?

Answer options

• A. A/B testing
• B. Canary release
• C. Shadow deployment
• D. Blue/green deployment

Correct answer: C

Explanation

Shadow deployment is the ideal strategy because it duplicates production traffic and sends it to the new experimental model to evaluate its performance without impacting the live user experience or the active model's responses. Other deployment methods like A/B testing, Canary release, and Blue/green deployment expose some or all users to the new model's predictions, which violates the requirement to not affect live traffic and only have one model serve requests.