IAPP Artificial Intelligence Governance Professional (AIGP) — Question 130
CASE STUDY -
Please use the following to answer the next question:
A leading insurance provider that offers a range of coverage options to individuals has decided to utilize AI to streamline and improve its customer acquisition and underwriting process, including the accuracy and efficiency of pricing policies. The company has engaged a cloud provider to utilize and fine-tune its pre-trained, general purpose large language model (“LLM”).
The company intends to use its historical customer data – including applications, policies and claims – and proprietary pricing and risk strategies to provide an initial qualification assessment of potential customers, which would then be routed to a human underwriter for final review.
The company and the cloud provider have completed training and testing the LLM, performed a readiness assessment, and made the decision to deploy the LLM into production. They have designated an internal compliance team to monitor the model during the first month, specifically to evaluate the accuracy, fairness and reliability of its output.
After the first month in production, the company realizes that the LLM declines a higher percentage of women’s applications.
During the first month when the company monitors the model for bias, it is most important to:
Answer options
- A. Continue disparity testing.
- B. Provide regular awareness training.
- C. Analyze the quality of the training and testing data.
- D. Document the results of final decisions made by the human underwriter.
Correct answer: A
Explanation
The correct answer is A because continuing disparity testing is essential to identify and measure bias in the model's decision-making. The other options, while important in their own contexts, do not directly address the immediate need to evaluate and mitigate bias within the LLM's outputs.