AWS Certified Machine Learning – Specialty — Question 47

A gaming company has launched an online game where people can start playing for free, but they need to pay if they choose to use certain features. The company needs to build an automated system to predict whether or not a new user will become a paid user within 1 year. The company has gathered a labeled dataset from 1 million users.
The training dataset consists of 1,000 positive samples (from users who ended up paying within 1 year) and 999,000 negative samples (from users who did not use any paid features). Each data sample consists of 200 features including user age, device, location, and play patterns.
Using this dataset for training, the Data Science team trained a random forest model that converged with over 99% accuracy on the training set. However, the prediction results on a test dataset were not satisfactory
Which of the following approaches should the Data Science team take to mitigate this issue? (Choose two.)

Answer options

• A. Add more deep trees to the random forest to enable the model to learn more features.
• B. Include a copy of the samples in the test dataset in the training dataset.
• C. Generate more positive samples by duplicating the positive samples and adding a small amount of noise to the duplicated data.
• D. Change the cost function so that false negatives have a higher impact on the cost value than false positives.
• E. Change the cost function so that false positives have a higher impact on the cost value than false negatives.

Correct answer: C, D

Explanation

Option C is correct because generating more positive samples can help balance the dataset and improve model performance on minority classes. Option D is also valid as adjusting the cost function to prioritize reducing false negatives can lead to better prediction for users who are likely to pay. Options A and B do not address the class imbalance issue, and option E would not help in this scenario since false negatives are more critical to capture accurately.