AWS Certified Big Data – Specialty — Question 49

A large grocery distributor receives daily depletion reports from the field in the form of gzip archives od CSV files uploaded to Amazon S3. The files range from 500MB to 5GB. These files are processed daily by an EMR job.
Recently it has been observed that the file sizes vary, and the EMR jobs take too long. The distributor needs to tune and optimize the data processing workflow with this limited information to improve the performance of the
EMR job.
Which recommendation should an administrator provide?

Answer options

• A. Reduce the HDFS block size to increase the number of task processors.
• B. Use bzip2 or Snappy rather than gzip for the archives.
• C. Decompress the gzip archives and store the data as CSV files.
• D. Use Avro rather than gzip for the archives.

Correct answer: B

Explanation

Choosing option B is correct because bzip2 or Snappy can offer better compression ratios and faster decompression speeds compared to gzip, which can enhance the performance of the EMR job. Option A is incorrect as reducing the HDFS block size could lead to increased overhead and is not guaranteed to improve job performance. Option C is not optimal because storing uncompressed CSV files would likely increase storage costs and processing time. Option D, while Avro is efficient, does not address the immediate issue of compression performance compared to gzip.