Google Cloud Professional Machine Learning Engineer — Question 319

You deployed a conversational application that uses a large language model (LLM). The application has 1,000 users. You collect user feedback about the verbosity and accuracy of the model 's responses. The user feedback indicates that the responses are factually correct but users want different levels of verbosity depending on the type of question. You want the model to return responses that are more consistent with users' expectations, and you want to use a scalable solution. What should you do?

Answer options

• A. Implement a keyword-based routing layer. If the user's input contains the words "detailed" or "description," return a verbose response. If the user's input contains the word "fact." re-prompt the language model to summarize the response and return a concise response.
• B. Ask users to provide examples of responses with the appropriate verbosity as a list of question and answer pairs. Use this dataset to perform supervised fine tuning of the foundational model. Re-evaluate the verbosity of responses with the tuned model.
• C. Ask users to indicate all scenarios where they expect concise responses versus verbose responses. Modify the application 's prompt to include these scenarios and their respective verbosity levels. Re-evaluate the verbosity of responses with updated prompts.
• D. Experiment with other proprietary and open-source LLMs. Perform A/B testing by setting each model as your application's default model. Choose a model based on the results.

Correct answer: A

Explanation

Option A is correct because it directly addresses the users' request for different levels of verbosity based on specific keywords, providing a quick and scalable solution. Other options like B and C involve more complex processes of user input collection and model tuning, which may take more time and resources. Option D, while valuable for model evaluation, does not specifically target the issue of adapting verbosity to user preferences.