Databricks Certified Data Engineer Professional — Question 147

Which statement regarding Spark configuration on the Databricks platform is true?

Answer options

• A. The Databricks REST API can be used to modify the Spark configuration properties for an interactive cluster without interrupting jobs currently running on the cluster.
• B. Spark configurations set within a notebook will affect all SparkSessions attached to the same interactive cluster.
• C. When the same Spark configuration property is set for an interactive cluster and a notebook attached to that cluster, the notebook setting will always be ignored.
• D. Spark configuration properties set for an interactive cluster with the Clusters UI will impact all notebooks attached to that cluster.

Correct answer: D

Explanation

The correct answer is D because Spark configuration properties set in the Clusters UI are applied at the cluster level, impacting all attached notebooks. Option A is incorrect as the REST API does not allow changes without job interruption. Option B is false because notebook settings can override cluster settings. Option C is misleading since notebook settings can take precedence over cluster settings in certain cases.