AWS Certified Data Engineer – Associate (DEA-C01) — Question 62

A data engineer needs to build an extract, transform, and load (ETL) job. The ETL job will process daily incoming .csv files that users upload to an Amazon S3 bucket. The size of each S3 object is less than 100 MB.
Which solution will meet these requirements MOST cost-effectively?

Answer options

• A. Write a custom Python application. Host the application on an Amazon Elastic Kubernetes Service (Amazon EKS) cluster.
• B. Write a PySpark ETL script. Host the script on an Amazon EMR cluster.
• C. Write an AWS Glue PySpark job. Use Apache Spark to transform the data.
• D. Write an AWS Glue Python shell job. Use pandas to transform the data.

Correct answer: D

Explanation

The correct answer is D, as AWS Glue Python shell jobs using pandas are well-suited for lightweight transformations of smaller datasets like the ones described. Options A and B involve more complex setups with higher operational costs, and option C, while effective, may not be as cost-efficient for the specific requirements of processing small .csv files.