Databricks Certified Data Engineer Professional — Question 182

In order to prevent accidental commits to production data, a senior data engineer has instituted a policy that all development work will reference clones of Delta Lake tables. After testing both DEEP and SHALLOW CLONE, development tables are created using SHALLOW CLONE.

A few weeks after initial table creation, the cloned versions of several tables implemented as Type 1 Slowly Changing Dimension (SCD) stop working. The transaction logs for the source tables show that VACUUM was run the day before.

Which statement describes why the cloned tables are no longer working?

Answer options

• A. Because Type 1 changes overwrite existing records, Delta Lake cannot guarantee data consistency for cloned tables.
• B. Running VACUUM automatically invalidates any shallow clones of a table; DEEP CLONE should always be used when a cloned table will be repeatedly queried.
• C. The data files compacted by VACUUM are not tracked by the cloned metadata; running REFRESH on the cloned table will pull in recent changes.
• D. The metadata created by the CLONE operation is referencing data files that were purged as invalid by the VACUUM command.

Correct answer: D

Explanation

The correct answer is D because the metadata from the SHALLOW CLONE operation references the original data files, which were deleted during the VACUUM process. While Type 1 changes do overwrite records, they do not cause the cloned tables to stop working. VACUUM does not invalidate shallow clones, but it removes the data files that those clones rely on, making them unusable.