Databricks Certified Data Engineer Associate — Question 138

A data engineer works for an organization that must meet a stringent Service Level Agreement (SLA) that demands minimal runtime errors and high availability for its data processing pipelines. The data engineer wants to avoid the operational overhead of managing and tuning clusters.

Which architectural solution will meet the requirements?

Answer options

• A. Use an auto-scaling cluster configured and monitored by the user.
• B. Implement a hybrid approach with scheduled batch jobs on custom cloud VMs.
• C. Deploy a dedicated, manually managed cluster optimized by in-house IT staff.
• D. Utilize Databricks serverless compute that automatically optimizes resources and abstracts cluster management.

Correct answer: D

Explanation

The correct answer is D because Databricks serverless compute handles resource optimization and cluster management automatically, aligning perfectly with the need for high availability and minimal runtime errors. Options A and C require manual oversight and tuning, which adds operational overhead, while B does not provide the high availability needed for stringent SLAs.