Health Insurers' AI Use Erodes Provider Trust
Health insurers are aggressively adopting AI to cut costs, but providers report this is undermining trust due to a lack of transparency in automated decisions. The trend is fueling a push for health systems to implement stronger AI governance, explainability, and control frameworks. This comes as Mayo Clinic deploys AI to predict diseases years before symptoms appear, highlighting both the potential of AI and the critical need for robust data governance.
- UnitedHealth Group faces a class-action lawsuit for allegedly using an AI model called nH Predict, which is claimed to have a 90% error rate, to deny necessary care to elderly patients under Medicare Advantage plans. The lawsuit alleges this practice forces patients to pay out-of-pocket or forgo care, and that the company knows only a very small fraction of policyholders (about 0.2%) will appeal the denials. - Cigna is also facing a class-action lawsuit in California for allegedly using its "PxDx" algorithm to automatically deny large batches of claims without individual physician review. Over a two-month period, Cigna doctors reportedly rejected over 300,000 payment requests, spending an average of just 1.2 seconds on each case. - In response to the growing use of AI, the Centers for Medicare & Medicaid Services (CMS) has clarified that while AI can assist in coverage determinations, it cannot be the sole basis for denying care. The guidance emphasizes that medical necessity must be based on an individual's specific circumstances, not just algorithmic predictions. - The push for greater regulation is also happening at the state level, with states like Alabama, New York, and Texas introducing bills that would require insurers to publicly disclose their use of AI algorithms for utilization review. These proposals would also mandate that insurers submit the algorithms and their training data to state insurance departments for oversight. - For data professionals in healthcare, implementing data observability is crucial for ensuring the quality, security, and reliability of the data powering analytics and AI models. This goes beyond simple monitoring to provide a holistic understanding of data health across complex, interconnected systems, which is vital in a regulated environment like healthcare. - Analytics engineering best practices, particularly using tools like dbt (data build tool), are becoming standard for healthcare data teams to build more reliable and maintainable data pipelines. By integrating software engineering patterns like version control, automated testing, and documentation directly into the analytics workflow, teams can create a single source of truth and improve the trustworthiness of data used for both clinical and operational decision-making.
