Jerusalem Post flags AI bias tool

Johns Hopkins University researchers and U.S. Food and Drug Administration collaborators say a new auditing tool is aimed at a basic problem in medical AI: a model can look accurate in testing while learning the wrong lesson from the data. The tool, called G-AUDIT, examines training datasets for attributes that may push a model toward shortcuts rather than clinically meaningful signals. The work was published in *npj Digital Medicine*, and was described this week by the *Jerusalem Post* and Johns Hopkins. Mathias Unberath, a Johns Hopkins expert in AI-assisted medicine and a senior author on the work, said predictive performance alone does not show what a model is actually using to make a decision. He said the tool is designed to surface metadata or dataset features that pose the highest risk of bias before those patterns are baked into deployed systems. ### How can a medical AI model be “right” for the wrong reason? (hub.jhu.edu) The Johns Hopkins team said medical models often learn by association, which means they can pick up signals that correlate with disease labels inside a training set but do not reflect biology or patient need. G-AUDIT looks for those associations in the data itself, rather than waiting to discover them only after a model has been built and tested. (hub.jhu.edu) Unberath gave examples of shortcut learning that can distort care. In one case cited by the *Jerusalem Post*, an AI system linked clinician skin markings with malignant lesions and produced 40% more false positives when those markings appeared in scans. In another, a model trained on images from two clinics could associate ruler presence or camera quality with cancer risk because those features tracked where the images came from. (hub.jhu.edu) ### What does G-AUDIT actually do? Johns Hopkins said G-AUDIT stands for Generalized Attribute Utility and Detectability-Induced bias Testing. The framework is described as modality-agnostic, meaning it is meant to work across different kinds of medical data, and it ranks attributes that are likely to drive flawed predictions. The goal is to help researchers generate targeted hypotheses about which variables may be introducing bias in training or testing data. (jpost.com) Mitchell Pavlak, a Johns Hopkins PhD student and co-author, said the shift is from checking model behavior after the fact to analyzing the dataset upfront to identify likely problems. That matters because a model can appear to generalize inside familiar data and then fail when used in a different clinic or with a different imaging setup. ### Why does regulation come up in the same conversation? (hub.jhu.edu) A separate study led by Nanyang Technological University, Singapore, found a gap between developers’ sense of responsibility and their familiarity with the rules governing medical AI. The survey covered 122 medical AI developers from Singapore, China, Hong Kong and the United Kingdom, and found that 57% were aware of at least one regulatory framework while 67% of their organizations had adopted none. (hub.jhu.edu) The findings were also reported in *npj Digital Medicine*. Wilson Goh of NTU’s Lee Kong Chian School of Medicine said the gap is concerning because developers are positioned to assess data quality, model limits and risks such as bias and hallucinations. NTU said limited awareness of safeguards such as the EU AI Act or Singapore’s AI in Healthcare Guidelines could affect patient outcomes and public trust when tools reach clinics. (medicalxpress.com) ### So what should readers take from these two reports together? The two reports point to different failure points in the same pipeline. Johns Hopkins and FDA researchers are focused on whether the dataset contains cues that can mislead a model even when accuracy looks strong. NTU researchers are focused on whether the people building those systems understand the frameworks meant to govern safety, privacy and deployment. That connection is an inference based on the two studies’ findings, not a direct quote from either team. (medicalxpress.com) The next place to watch is *npj Digital Medicine*, where both the dataset-bias work and the developer-regulation survey were published, and the Johns Hopkins team said G-AUDIT is intended to help researchers and regulators assess medical AI before real-world clinical use. (hub.jhu.edu)