Databricks Certified Data Engineer Professional — Question 5

Each configuration below is identical to the extent that each cluster has 400 GB total of RAM, 160 total cores and only one Executor per VM.
Given a job with at least one wide transformation, which of the following cluster configurations will result in maximum performance?

Answer options

• A. • Total VMs; 1 • 400 GB per Executor • 160 Cores / Executor
• B. • Total VMs: 8 • 50 GB per Executor • 20 Cores / Executor
• C. • Total VMs: 16 • 25 GB per Executor • 10 Cores/Executor
• D. • Total VMs: 4 • 100 GB per Executor • 40 Cores/Executor
• E. • Total VMs:2 • 200 GB per Executor • 80 Cores / Executor

Correct answer: C

Explanation

The correct answer is C because having 16 VMs allows for greater parallel processing, which is crucial for wide transformations. Options A, D, and E have fewer VMs, which limits the ability to distribute the workload effectively. Option B, while having more VMs, allocates too little memory and cores per executor to maximize performance.