Google Cloud Professional Data Engineer — Question 52

After migrating ETL jobs to run on BigQuery, you need to verify that the output of the migrated jobs is the same as the output of the original. You've loaded a table containing the output of the original job and want to compare the contents with output from the migrated job to show that they are identical. The tables do not contain a primary key column that would enable you to join them together for comparison.
What should you do?

Answer options

• A. Select random samples from the tables using the RAND() function and compare the samples.
• B. Select random samples from the tables using the HASH() function and compare the samples.
• C. Use a Dataproc cluster and the BigQuery Hadoop connector to read the data from each table and calculate a hash from non-timestamp columns of the table after sorting. Compare the hashes of each table.
• D. Create stratified random samples using the OVER() function and compare equivalent samples from each table.

Correct answer: C

Explanation

The correct answer is C because it allows for a comprehensive comparison of the data by generating hashes from sorted non-timestamp columns, ensuring that differences can be detected. Options A and B are inadequate as they only compare random samples, which may not represent the complete data. Option D does not resolve the issue of lacking a primary key for a thorough comparison.