AI hallucinations still dangerous
An AI tool recently hallucinated a patient’s type‑2 diabetes diagnosis, inventing medications and hospital details — a stark real‑world failure in clinical automation (x.com). Experts warn these errors show verification must remain central as AI moves from drafting to decision‑support in medicine (palatinate.org.uk).
Fortune’s reporting, published July 20, 2025, says a London patient in his mid‑20s found an entry in his record explicitly labelled “generated by Annie AI.” (uk.news.yahoo.com) The published records reviewed by reporters show the AI summary attributed chest pain and “likely angina due to coronary artery disease” to the patient and recorded a Type‑2 diabetes diagnosis with named medications and dosages that did not match other clinical notes. (uk.news.yahoo.com) The system implicated is Anima Health’s ambient scribe, marketed as Anima Scribe or “Annie,” which the company lists on its product pages as registered with the MHRA as a Class I medical device and compliant with DTAC, GDPR and Cyber Essentials. (animahealth.com) Anima’s own blog says the company is pursuing MHRA Class 2a reclassification for its AVT/summarisation product, a step it describes as part of meeting NHS England AVT requirements. (animahealth.com) An NHS spokesperson quoted in the reporting said the GP practice used “limited supervised AI” and characterised the event as a “one‑off case of human error,” a claim that has been cited in follow‑up coverage. (pslhub.org) NHS England guidance for AI‑enabled ambient scribing requires MHRA registration and proportional risk assessment, while the MHRA’s PARD notes registration is not an approval—two regulatory points central to the scrutiny triggered by the case. (england.nhs.uk) (pard.mhra.gov.uk) A February 2026 whistleblower report and subsequent press pieces renewed scrutiny of automated summaries in NHS records and flagged ongoing questions about human oversight, reporting channels and legal liability for AI‑driven errors. (abcmoney.co.uk)
