Databricks Certified Associate Developer for Apache Spark — Question 54

Which of the following Spark properties is used to configure whether skewed partitions are automatically detected and subdivided into smaller partitions when joining two DataFrames together?

Answer options

• A. spark.sql.adaptive.skewedJoin.enabled
• B. spark.sql.adaptive.coalescePartitions.enable
• C. spark.sql.adaptive.skewHints.enabled
• D. spark.sql.shuffle.partitions
• E. spark.sql.shuffle.skewHints.enabled

Correct answer: A

Explanation

The correct answer, A, specifically enables the detection and subdivision of skewed partitions during DataFrame joins. The other options do not relate to skew detection; B focuses on coalescing partitions, C addresses skew hints without automatic detection, D deals with shuffle partitions configuration, and E is related to skew hints rather than skewed join management.