New Focus on Clinical AI Governance and Safety
As AI gets embedded in bedside care, there's a growing focus on clinical AI governance to ensure safety, validate models, and mitigate bias. The conversation is also shifting toward next-gen "reasoning AI" that could transform complex decision-making, increasing the need for robust clinical oversight.
The U.S. Food and Drug Administration (FDA) is actively developing a regulatory framework for AI/ML-enabled medical devices, focusing on a "total product lifecycle" approach. This includes issuing draft guidance on predetermined change control plans (PCCPs), which would allow manufacturers to implement pre-approved modifications to their AI algorithms without needing a new submission for each update. As of December 2024, the FDA had authorized 1,016 AI/ML-enabled medical devices. A major focus of clinical AI governance is the mitigation of bias, which can arise from underrepresented data, flawed algorithms, or human developers, potentially worsening existing healthcare disparities. Best practices to address this include using diverse and representative datasets, conducting regular bias audits with fairness metrics, and implementing "human-in-the-loop" reviews for high-stakes clinical decisions. Pre-processing strategies like data balancing and in-processing techniques such as fairness-aware algorithms are also key mitigation strategies. For ICU nurses transitioning to informatics, practical experience is key. Many start by becoming an EHR "super user," participating in system rollouts, or taking on informal project work related to workflow assessments. These experiences build connections with IT staff and provide a practical understanding of health IT systems. Entry-level roles often include positions like EMR trainer, project coordinator, or clinical liaison, which serve as a bridge between clinical expertise and technical responsibilities. EHR giant Epic Systems is embedding AI and machine learning across its platform, which is used by over 3,000 hospitals and holds records for more than 325 million patients. New tools like "Emmie" and "Art" aim to assist patients in understanding test results and help clinicians reduce administrative work. Epic's AI Charting feature, developed with Microsoft, ambiently listens to patient visits to draft clinical notes and queue up orders, aiming to reduce documentation burden. The transition to nursing informatics requires a blend of clinical expertise and new technical and analytical skills. Certifications like the ANCC Nursing Informatics Certification (NI-BC) are highly valued, though they often require at least two years of informatics experience. Employers look for skills in project management, data analysis, and effective communication to bridge the gap between clinical end-users and technical teams. Effective AI governance in healthcare requires a multidisciplinary approach, bringing together stakeholders from clinical leadership, informatics, data science, bioethics, and legal departments. The goal is to create a clear framework for accountability that oversees the entire lifecycle of an AI tool, from pre-implementation validation to continuous post-deployment monitoring. This structure ensures that AI tools are used safely, ethically, and in compliance with regulations like HIPAA.
