AWS Certified Machine Learning – Specialty — Question 173

A data scientist is evaluating a GluonTS on Amazon SageMaker DeepAR model. The evaluation metrics on the test set indicate that the coverage score is 0.489 and 0.889 at the 0.5 and 0.9 quantiles, respectively.
What can the data scientist reasonably conclude about the distributional forecast related to the test set?

Answer options

• A. The coverage scores indicate that the distributional forecast is poorly calibrated. These scores should be approximately equal to each other at all quantiles.
• B. The coverage scores indicate that the distributional forecast is poorly calibrated. These scores should peak at the median and be lower at the tails.
• C. The coverage scores indicate that the distributional forecast is correctly calibrated. These scores should always fall below the quantile itself.
• D. The coverage scores indicate that the distributional forecast is correctly calibrated. These scores should be approximately equal to the quantile itself.

Correct answer: D

Explanation

The correct answer is D because a well-calibrated forecast should have coverage scores that are close to the quantile levels they represent. Options A, B, and C incorrectly describe the expected relationship between coverage scores and quantiles, indicating miscalibration.