AI in hospitals: liability and bias flagged

Artificial intelligence tools are moving into hospitals faster than rules for bias, liability, and safe use are catching up. (fda.gov) (jamanetwork.com) In hospitals, these systems are sold as software that predicts risk, drafts notes, summarizes charts, or suggests diagnoses and treatments. The Food and Drug Administration says software can be a medical device when it makes predictions, recommendations, or decisions that affect care. (fda.gov) The Food and Drug Administration has tried to address software that changes after launch, because machine-learning systems can improve or drift as they absorb new data. In August 2025, the agency finalized guidance on a “predetermined change control plan,” which lets companies spell out in advance what updates they expect to make and how they will validate them. (fda.gov) That still leaves a bedside problem: a hospital doctor may see an artificial intelligence suggestion in the middle of a packed shift and decide whether to trust it in seconds. A March 21, 2025, JAMA Health Forum viewpoint said hospitals are adopting assistive artificial intelligence faster than laws and legal standards are evolving, while physicians remain the people most likely to be blamed after a bad outcome. (jamanetwork.com) Researchers call one risk “automation bias,” which is the tendency to follow a machine’s advice even when it is wrong. A 2023 Scientific Reports study on mammography said medical professionals are prone to follow incorrect algorithmic suggestions, especially when they have limited ability to interrogate the output. (nature.com) Evidence on benefit is mixed. In a randomized clinical trial published in JAMA Network Open in October 2024, 50 physicians using a large language model scored 76% on diagnostic performance versus 74% with conventional resources, a difference that was not statistically significant. (jamanetwork.com) Some tools do appear to save time on paperwork. A quality-improvement study published in JAMA Network Open in May 2025 evaluated an ambient artificial intelligence documentation platform for clinicians, part of a push to use software as a digital scribe rather than a digital diagnostician. (jamanetwork.com) Hospitals are also adopting these systems during a financing squeeze, and ownership structure shapes how new software gets deployed. A Health Affairs analysis published in May 2024 said private-equity acquisitions in health care delivery nearly tripled from 2010 to 2020, and a JAMA Internal Medicine survey said physicians worry private-equity ownership can affect staffing, management pressure, and workplace conditions. (healthaffairs.org) (jamanetwork.com) The market is no longer small. The Food and Drug Administration’s artificial-intelligence-enabled device list says it is tracking authorized products already on the United States market, most of them in imaging, even as newer generative tools spread through documentation and triage workflows. (fda.gov) Researchers are also measuring how fast health care companies are moving. A JAMA Health Forum study using United States Census Bureau business survey data examined artificial intelligence use from September 2023 to May 2025 and found adoption rising in health care, though still below some other sectors. (jamanetwork.com) The near-term fight is not whether hospitals will use artificial intelligence, but who carries the risk when the software is wrong and the clinician clicks “accept.” Federal regulators are building update rules, journals are publishing bias warnings, and hospitals are still deciding how much judgment to delegate to the screen. (fda.gov) (jamanetwork.com)