NEJM Clinician flags ambient AI scribe risks
- NEJM Clinician published a May 6, 2026 commentary reviewing ambient AI scribes and flagging accuracy, hallucination, and omission risks for clinicians and regulators. (clinician.nejm.org) - Sixty percent of approved AI scribes recorded a different drug in the auditor’s tests, Auditor General Shelley Spence said in her May 12 report. (thetrillium.ca) - The Auditor General’s special report is available on auditor.on.ca; Supply Ontario and provincial ministries are the next named participants. (auditor.on.ca)
The New England Journal of Medicine’s NEJM Clinician editors published a commentary on May 6, 2026, that reviewed recent evidence on ambient AI scribes and warned they produce drafts, not finished clinical notes. (clinician.nejm.org) The Veterans Health Administration evaluated 11 commercial AI scribe tools against human note‑takers in five standardized primary‑care cases and found consistently lower quality scores for AI‑generated notes, investigators reported in the Annals of Internal Medicine. (thetrillium.ca) The Office of the Auditor General of Ontario released a special report on May 12, 2026, that tested AI scribe products and reported hallucinations, inaccuracies and gaps in procurement testing by Supply Ontario. (auditor.on.ca) ### What specific accuracy issues did NEJM Clinician highlight? NEJM Clinician’s May 6 summary said AI scribes create “decent drafts — not finished notes,” and that their performance depends on audio quality and context. (clinician.nejm.org) The commentary noted risks of errors and omissions in domains such as organization, thoroughness and clinical usefulness and recommended clinicians treat AI outputs as drafts requiring review. (medicalxpress.com) ### What did the VHA/Annals evaluation find when comparing AI to human notes? Ashok Reddy and VHA co‑authors reported that human‑generated notes scored higher than AI across all five cases on the modified Physician Documentation Quality Instrument (PDQI‑9). (auditor.on.ca) The VHA analysis found the largest performance gap in an acute low‑back‑pain case, where human notes scored about 43.8 versus 20.3 for AI notes, and the authors urged rigorous, ongoing quality evaluation before broad deployment. (clinician.nejm.org) ### What did Ontario’s auditor actually find in vendor testing? Auditor General Shelley Spence said Supply Ontario tested 20 pre‑qualified AI scribe vendors using simulated recordings and found widespread errors — including hallucinated treatment suggestions and missing mental‑health details in many outputs. (clinician.nejm.org) Spence’s report noted that 12 of 20 systems captured a different drug than the one prescribed in at least one test, and that testing procedures and privacy/security documentation were incomplete. (medicalxpress.com) ### What safeguards do clinicians and researchers recommend? Nature and other expert reviews have urged provenance tracking, audit trails and model‑version documentation so health systems can trace what model produced each note and when. (medicalxpress.com) Researchers who led the head‑to‑head evaluations and NEJM Clinician’s editors both recommended local validation and clinician review of AI‑drafted notes before those notes enter the legal medical record. (auditor.on.ca) ### How has the Ontario government responded? CBC and Radio‑Canada reported that provincial officials said the errors appeared in testing and that procurement and rollout remain under review; the province has not announced a formal suspension. (thetrillium.ca) ### What are the next concrete steps and milestones? The Auditor General tabled the special report on May 12, 2026; Supply Ontario and the named ministries are listed in the report and are the next participants expected to respond publicly to the findings. (nature.com) (auditor.on.ca) (cbc.ca) (medicalxpress.com)
