Hospitals shift to AI governance

Hospitals are starting to treat artificial intelligence less like a software purchase and more like a medication order: before it reaches a patient, someone has to decide where it belongs, what could go wrong, and who signs off. One April 9 report says health system technology chiefs are redesigning rollout plans around governance, security controls, and infrastructure instead of simple adoption. (distilinfo.com) The first big use case is ambient documentation, which means software listens to a doctor-patient conversation and drafts the visit note. Parkview Health executive Mark Mabus said the appeal is simple: clinicians finish notes faster and spend less time typing. (distilinfo.com) That convenience creates a new checklist that old hospital purchasing teams were not built for. Mabus said hospitals now ask where audio is processed, whether protected health information is retained, and who validates the output before a tool goes live. (distilinfo.com) Protected health information is the legally guarded record of your care, like diagnoses, lab results, and billing details tied to your name. The United States Department of Health and Human Services says the Office for Civil Rights enforces the Health Insurance Portability and Accountability Act, which is the main federal privacy rule hospitals have to navigate when artificial intelligence touches patient data. (hhs.gov) Hospitals are also drawing a line between drafting and deciding. In the Parkview example, the software can write a note, but the physician still has to edit it and sign it before it becomes part of the chart. (distilinfo.com) That human check is not just a legal comfort blanket; it is a quality filter. Mabus said some tools that looked strong in demos created extra work in practice, including cases where a doctor expected three lines and got nine paragraphs instead. (distilinfo.com) The pressure to move fast is real even with those guardrails. A Qventus study published April 9 found that 42 percent of surveyed health systems were actively deploying artificial intelligence across multiple use cases, but only 4 percent said they had scaled it with measurable outcomes. (healthcareitnews.com) The same study shows why governance is replacing pilot fever. Researchers surveyed more than 60 senior health care technology leaders, and 25 percent said they lacked a clear process for benchmarking artificial intelligence performance while 45 percent said scaling pilots was difficult. (healthcareitnews.com) Consumer tools are one reason hospitals are tightening the screws. A study covered on April 9 found Google’s Artificial Intelligence Overviews answered about 9 out of 10 queries correctly, but because Google handles roughly five trillion searches a year, that still translated into tens of millions of wrong answers every hour. (mobilesyrup.com) The same analysis found more than half of the accurate answers were “ungrounded,” which means the links shown with the answer did not fully support what the summary said. That is exactly the kind of mismatch a hospital cannot wave through when the topic is a medication dose, a discharge plan, or a cancer screening result. (mobilesyrup.com) Federal regulators are already moving in the same direction as hospital governance teams. The Food and Drug Administration says its public list of authorized artificial intelligence-enabled devices is meant to improve transparency, and it says future updates will identify devices that use large language model functionality. (fda.gov) The Food and Drug Administration, Health Canada, and the United Kingdom’s medicines regulator have also published transparency principles that focus on intended use, performance, logic, explainability, and the performance of the human-artificial intelligence team. Hospitals are now building those same questions into internal approval before a tool ever reaches a clinic floor. (fda.gov)