Regulated health needs lineage and drift controls

A hospital can tolerate a calculator that is slow. It cannot tolerate a language model that invents a dosage, drops an allergy, or quietly changes behavior after a software update. (fda.gov) That is why regulators treat medical artificial intelligence less like a chatbot and more like a pacemaker manual: every version, training source, test result, and update path has to be traceable. The United States Food and Drug Administration says artificial intelligence devices need management across the full product life cycle, not just at launch. (fda.gov) “Lineage” is the paper trail for an artificial intelligence system. It means a company can show which data trained the model, which code built it, which version was shipped, and which evidence supported each change. (fda.gov) “Drift” is what happens when a model that worked on yesterday’s patients starts seeing today’s patients and the pattern shifts. The Food and Drug Administration is already funding postmarket tools to detect input changes and watch output performance after these systems reach clinics. (fda.gov) The reason this is suddenly urgent is that large language models do not just make arithmetic mistakes. A 2025 study in npj Digital Medicine built a framework to score hallucinations and omissions in medical summarization because both kinds of errors can distort the clinical record. (nature.com) Regulators are also preparing for a world where approved models keep changing. In August 2025, the Food and Drug Administration finalized guidance for a “predetermined change control plan,” which is a pre-agreed map of what can be modified later without starting the entire review from scratch. (fda.gov) That only works if the manufacturer can describe the update before it happens. The same guidance asks for the exact modifications planned, the method used to implement them, and the assessment used to keep the device safe and effective after the change. (fda.gov) Transparency is becoming a separate regulatory demand, not a nice extra. The Food and Drug Administration, Health Canada, and the United Kingdom Medicines and Healthcare products Regulatory Agency jointly said machine-learning medical devices should communicate information that could affect patient risk and outcomes to the people using them. (fda.gov) There is also a data problem hiding underneath the model problem. A 2024 review of 692 Food and Drug Administration approved artificial intelligence and machine learning devices found that 81.6% did not report study-subject age, 99.1% provided no socioeconomic data, and only 3.6% reported race or ethnicity. (nature.com) That means “worked in testing” may not tell a hospital much about how the tool will behave on its own population. If the training data, validation data, and live patients do not match, the model can drift even when the code stays the same. (nature.com) The practical shift in regulated health is simple: do not buy a model, buy a control system around the model. In this market, the winning vendors are the ones that can show provenance, monitoring, rollback plans, and an audit trail sturdy enough for an inspector to follow line by line. (fda.gov)