Databricks Certified Generative AI Engineer Associate — Question 90

A generative AI engineer is deploying an AI agent authored with MLflow's ChatAgent interface for a retail company's customer support system on Databricks. The agent must handle thousands of inquiries daily, and the engineer needs to track its performance and quality in real-time to ensure it meets service-level agreements.

Which metrics are automatically captured by default and made available for monitoring when the agent is deployed using the Mosaic AI Agent Framework?

Answer options

• A. Operational metrics like request volume, latency, and errors
• B. Quality metrics like correctness and guideline adherence
• C. Both operational and quality metrics
• D. No metrics are automatically captured

Correct answer: A

Explanation

The correct answer is A because the Mosaic AI Agent Framework is designed to automatically capture operational metrics such as request volume, latency, and errors, which are crucial for real-time performance monitoring. Options B and C refer to quality metrics that may not be automatically tracked, while D is incorrect as it states that no metrics are captured, which contradicts the functionality of the framework.