Google Cloud Professional Data Engineer — Question 204

You have thousands of Apache Spark jobs running in your on-premises Apache Hadoop cluster. You want to migrate the jobs to Google Cloud. You want to use managed services to run your jobs instead of maintaining a long-lived Hadoop cluster yourself. You have a tight timeline and want to keep code changes to a minimum. What should you do?

Answer options

• A. Move your data to BigQuery. Convert your Spark scripts to a SQL-based processing approach.
• B. Rewrite your jobs in Apache Beam. Run your jobs in Dataflow.
• C. Copy your data to Compute Engine disks. Manage and run your jobs directly on those instances.
• D. Move your data to Cloud Storage. Run your jobs on Dataproc.

Correct answer: D

Explanation

The correct answer is D because moving data to Cloud Storage and running jobs on Dataproc allows for a seamless migration of Spark jobs with minimal code changes. Option A involves significant changes to the codebase by converting to SQL, while B requires rewriting jobs in Apache Beam, which is not ideal for a tight timeline. Option C does not utilize managed services and requires more maintenance.