Databricks Certified Data Engineer Professional — Question 77

Which statement regarding Spark configuration on the Databricks platform is true?

Answer options

• A. The Databricks REST API can be used to modify the Spark configuration properties for an interactive cluster without interrupting jobs currently running on the cluster.
• B. Spark configurations set within a notebook will affect all SparkSessions attached to the same interactive cluster.
• C. Spark configuration properties can only be set for an interactive cluster by creating a global init script.
• D. Spark configuration properties set for an interactive cluster with the Clusters UI will impact all notebooks attached to that cluster.
• E. When the same Spark configuration property is set for an interactive cluster and a notebook attached to that cluster, the notebook setting will always be ignored.

Correct answer: D

Explanation

Option D is correct because when Spark configuration properties are set via the Clusters UI, they apply to all notebooks using that interactive cluster. Option A is incorrect as the REST API can modify configurations but may affect running jobs. Options B and E are inaccurate since notebook configurations may override cluster settings, and C misrepresents the flexibility of setting configurations in Databricks.