Insurers Wary of 'Black Box' AI

Regulatory bodies are closing in on the "black box" problem in insurance AI. In the U.S., the National Association of Insurance Commissioners (NAIC) has introduced a model bulletin, already adopted by 24 states, requiring insurers to document and be able to explain their AI systems to regulators. This framework mandates written AI programs, robust governance, and clear risk management controls. This regulatory push is a direct response to the inherent risks of opaque AI, where biased historical data can lead to discriminatory outcomes in pricing and claims decisions. For instance, if past data reflects systemic biases, an AI model could unfairly assign higher premiums or deny coverage to certain demographic groups, perpetuating those historical inequalities. The core issue is that without transparency, it's difficult to prove compliance with fair trade and anti-discrimination laws. Despite the compliance hurdles, AI adoption is surging, with AI deployments in the insurance sector jumping 87% year-over-year. Over 85% of insurers now utilize AI in their claims workflows, with projections showing the AI in insurance market could reach nearly $80 billion by 2032. This growth is driven by significant efficiency gains; some carriers report 80% faster processing times for low-severity claims and overall claims handling cost reductions between 25-40%. The "human-in-the-loop" (HITL) model provides a strategic solution, blending AI's data-processing power with human expertise. In this framework, AI handles routine tasks like data extraction and flags anomalies, while human adjusters, underwriters, and fraud specialists make the final, nuanced decisions. This approach is crucial for validating estimates, negotiating complex settlements, and handling scenarios that require empathy and contextual understanding—qualities an algorithm lacks. This collaborative model directly enhances accuracy and builds trust. Human oversight acts as a crucial check against algorithmic errors and model drift over time. For customers, the knowledge that a human expert is involved in critical decisions fosters confidence, while for regulators, it provides a clear line of accountability that is absent in a fully automated system. The industry is also seeing a rise in "agentic AI" systems, which can autonomously manage multi-step processes and are particularly suited for complex, data-heavy claims management. In late 2025, these advanced systems accounted for 21% of public AI deployments in insurance. This signals a shift from AI as a simple productivity tool to an operational system that runs core processes, with claims management being the primary focus for this technology. Ultimately, the trajectory is not toward replacing human adjusters but augmenting their capabilities. By automating the roughly 30% of an adjuster's time spent on low-value administrative work, AI frees them to concentrate on high-value activities like complex problem-solving and building policyholder relationships. This partnership leads to faster, fairer outcomes and is becoming a strategic necessity for staying competitive.