AI tools need auditing
Experts are warning that AI tools with embedded bias should be treated like contaminated medical products and routinely audited or pulled unless validated — a call that puts lab supervisors squarely in charge of algorithm governance. UNC research on AI trustworthiness reinforces the need for transparent validation, human oversight, and staff training before clinical deployment. (chiefhealthcareexecutive.com) (unc.edu)
Avishkar Sharma, director of artificial intelligence at Jefferson Health and an AI researcher at Stanford, told a HIMSS Global Health audience that biased clinical AI should be pulled “like a contaminated batch of propofol” and urged institutions to “be the chief ethical officer of your data.” (chiefhealthcareexecutive.com) Viviane Ito, a third‑year doctoral student at UNC’s School of Information and Library Science, is testing how patients interact with AI health information and was named a 2025 Google Ph.D. Fellow in Human‑Computer Interaction for her work on chatbot trustworthiness. (unc.edu) A multi‑institutional UNC analysis of more than 500 FDA‑authorized AI medical devices found roughly half lacked published clinical validation data, and the group reported FDA annual AI device authorizations rose from 2 in 2016 to about 69 per year. (news.unchealthcare.org) ECRI placed AI among the top patient‑safety hazards for 2025, and CEO Marcus Schabacker warned that large health systems often lack governance to oversee AI use and called for disclosure akin to “nutrition labels” for AI tools. (chiefhealthcareexecutive.com) The Organ Procurement and Transplantation Network moved to ban race‑inclusive eGFR calculations and approved wait‑time modifications starting Jan. 5, 2023; a subsequent quasi‑experimental analysis published in JAMA Internal Medicine reported short‑term increases in kidney transplant rates among Black candidates after those policy changes. (unos.org) Regulatory and consensus guidance is converging on lifecycle controls: FDA draft guidance now asks developers to describe postmarket monitoring and user training plans, the FUTURE‑AI consortium published 30 best‑practice items for trustworthy healthcare AI, and IHI and NHS guidance all call for clinician training tied to deployment. (fda.gov) Sharma recommended that hospitals form cross‑functional audit communities now rather than await federal rules, and UNC researchers explicitly urged manufacturers and regulators to conduct and publish clinical validation studies to support transparency before clinical rollout. (chiefhealthcareexecutive.com)
