AWS Certified Data Engineer – Associate (DEA-C01) — Question 227

A company uses AWS Glue Apache Spark jobs to handle extract, transform, and load (ETL) workloads. The company has enabled logging and monitoring for all AWS Glue jobs.

One of the AWS Glue jobs begins to fail. A data engineer investigates the error and wants to examine metrics for all individual stages within the job.

How can the data engineer access the stage metrics?

Answer options

• A. Examine the AWS Glue job and stage details in the Spark UI.
• B. Examine the AWS Glue job and stage metrics in Amazon CloudWatch.
• C. Examine the AWS Glue job and stage logs in AWS CloudTrail logs.
• D. Examine the AWS Glue job and stage details by using the run insights feature on the job.

Correct answer: A

Explanation

The correct answer is A because the Spark UI provides detailed insights into the job execution and its stages, which is essential for debugging. Option B is incorrect as CloudWatch primarily tracks high-level metrics rather than specific stage metrics. Option C is not suitable because AWS CloudTrail logs API calls rather than job execution metrics. Option D, while useful for general job insights, does not provide the detailed stage metrics needed for troubleshooting.