AWS Certified Machine Learning – Specialty — Question 207

A data scientist is training a large PyTorch model by using Amazon SageMaker. It takes 10 hours on average to train the model on GPU instances. The data scientist suspects that training is not converging and that resource utilization is not optimal.

What should the data scientist do to identify and address training issues with the LEAST development effort?

Answer options

• A. Use CPU utilization metrics that are captured in Amazon CloudWatch. Configure a CloudWatch alarm to stop the training job early if low CPU utilization occurs.
• B. Use high-resolution custom metrics that are captured in Amazon CloudWatch. Configure an AWS Lambda function to analyze the metrics and to stop the training job early if issues are detected.
• C. Use the SageMaker Debugger vanishing_gradient and LowGPUUtilization built-in rules to detect issues and to launch the StopTrainingJob action if issues are detected.
• D. Use the SageMaker Debugger confusion and feature_importance_overweight built-in rules to detect issues and to launch the StopTrainingJob action if issues are detected.

Correct answer: C

Explanation

The correct answer is C because the SageMaker Debugger's built-in rules specifically target issues like vanishing gradients and low GPU utilization, making it effective for diagnosing training problems. Options A and B focus on CPU metrics and custom metrics, which may not provide direct insights into GPU utilization issues, while D addresses different concerns that are not as relevant for the described situation.