Nature flags shaky AI training data
A Nature report found dozens of AI disease‑prediction models were trained on questionable data, suggesting model performance and provenance need scrutiny before clinical use. The same coverage included a clinically interpretable deep‑learning 'digital biopsy' for predicting early gastric‑cancer recurrence, illustrating both the promise and the provenance risks of medical AI. (nature.com, nature.com)
Medical artificial intelligence learns patterns from past patient records, much as a weather model learns from old storms. On April 15, Nature reported that 124 published disease-prediction studies relied on two public data sets whose origins could not be verified. (nature.com, medrxiv.org) Those two data sets, hosted on Kaggle and used for stroke and diabetes prediction, showed major gaps in basic provenance details such as when, where, why and how the records were collected. The medRxiv preprint said the data were “likely to be simulated or fabricated.” (medrxiv.org) The same preprint, by Alexander D. Gibson, Nicole M. White, Gary S. Collins and Adrian G. Barnett, found signs that three of the resulting prediction models had been used in clinical practice. It also found one model cited in a medical-device patent and 86 review articles citing models built on the suspect data. (medrxiv.org) A disease-prediction model is supposed to estimate a person’s odds of developing an illness from patterns in medical history, lab values or images. Nature said some of the affected models were built to estimate diabetes or stroke risk, and that at least two journals are investigating papers that used the data sets. (nature.com) That scrutiny lands as medical artificial intelligence moves from narrow tools toward systems that forecast many conditions at once. Nature Biotechnology reported on February 17 that newer models are being pitched to predict more than 1,000 diseases years in advance from health records and current health data. (nature.com) One example came in Nature Communications on April 15, when researchers published a “digital biopsy” system for gastric cancer, or stomach cancer, that reads routine pathology slides like a pathologist reading tissue under a microscope. The model, called Recurrence Stratification and Assessment, combined slide features with clinical variables to predict early recurrence after surgery. (nature.com) That gastric-cancer model was built on a retrospective multicenter cohort of 1,763 patients and then tested in two internal cohorts, two geographically distinct external cohorts and a post-hoc analysis of a prospective clinical-trial population. The paper reported area-under-the-curve scores from 0.843 to 0.887 and used Shapley Additive Explanations to show which tissue features drove risk estimates. (nature.com) The contrast is in the paper trail as much as the code. The gastric-cancer study listed a named public image source, TCGA-STAD in The Cancer Imaging Archive, and deposited its sequencing data in the National Genomics Data Center under accession HRA013404, with access controls tied to patient privacy. (nature.com) The authors of the medRxiv preprint said journals and repositories should require data-provenance reporting before publication. Nature’s report said the immediate question is not whether medical artificial intelligence can predict disease, but whether anyone can verify the records it learned from. (medrxiv.org, nature.com)
