Databricks Certified Data Engineer Professional — Question 100
Where in the Spark UI can one diagnose a performance problem induced by not leveraging predicate push-down?
Answer options
- A. In the Executor’s log file, by grepping for "predicate push-down"
- B. In the Stage’s Detail screen, in the Completed Stages table, by noting the size of data read from the Input column
- C. In the Storage Detail screen, by noting which RDDs are not stored on disk
- D. In the Delta Lake transaction log. by noting the column statistics
- E. In the Query Detail screen, by interpreting the Physical Plan
Correct answer: E
Explanation
The correct answer is E because the Query Detail screen provides insights into the Physical Plan, which can reveal whether predicate push-down optimizations are being applied. The other options do not directly address the performance impact of predicate push-down or are not relevant to diagnosing such issues in Spark's execution context.