Databricks Certified Associate Developer for Apache Spark — Question 139

The code block shown below should adjust the number of partitions used in wide transformations like join() to 32. Choose the response that correctly fills in the numbered blanks within the code block to complete this task.

Code block:

__1__(__2__, __3__)

Answer options

• A. 1. spark.conf.get 2. "spark.sql.shuffle.partitions" 3. "32"
• B. 1. spark.conf.set 2. "spark.default.parallelism" 3. 32
• C. 1. spark.conf.text 2. "spark.default.parallelism" 3. "32"
• D. 1. spark.conf.set 2. "spark.default.parallelism" 3. "32"
• E. 1. spark.conf.set 2. "spark.sql.shuffle.partitions" 3. "32"

Correct answer: E

Explanation

The correct answer is E because it uses 'spark.conf.set' to correctly configure the number of partitions for shuffling operations to 32 through the property 'spark.sql.shuffle.partitions'. Options A, B, C, and D have incorrect methods or property names that do not pertain to the required adjustment for shuffle partitions.