AI success hinges on clinician co‑design, not hype
Multiple pieces this week argue that AI projects fail unless clinicians co‑design them, regulators demand evidence, and tools are deployed in narrow, well‑defined tasks—examples include Qventus warnings and a recent FDA clearance for an AI gestational‑age ultrasound tool. The HIT Consultant piece stresses clinician workflow design, Isaac Kohane’s blog warns against 'doublethink' in AI training, and Diagnostic Imaging noted Butterfly’s FDA clearance as an example of narrowly scoped, image‑trained AI; meanwhile, Clinical Lab Products flagged an AI test predicting neoadjuvant response in breast cancer. Together these signals show the practical path for AI in diagnostics is cautious, clinician‑led integration and rigorous validation. (hitconsultant.net) (zaklab.org) (diagnosticimaging.com) (clpmag.com)
Most medical artificial intelligence fails for the same reason a fancy new cockpit fails if the pilots never helped place the buttons: the software may work, but the workflow breaks. Andrew Fisher, an anesthesiologist working with Qventus, wrote on April 10 that hospital projects stall when tools are dropped into perioperative care without clinician co-design. (hitconsultant.net) Perioperative care is the chain of work around surgery: testing before the operation, handoffs on the day, and follow-up after discharge. In that chain, one missed phone call or one unsigned clearance can cancel an operating room slot that took weeks to schedule. (hitconsultant.net) That is why the useful version of medical artificial intelligence is usually narrow. Instead of “replace the doctor,” the real products do one bounded job, like estimating fetal age from ultrasound images or flagging which cancer patient may respond to a treatment plan. (diagnosticimaging.com) (clpmag.com) A recent example came from Butterfly Network, which won Food and Drug Administration 510(k) clearance for a Gestational Age tool. The software estimates gestational age between 16 and 37 weeks and gives a result in less than two minutes after a three-step blind sweep ultrasound method. (diagnosticimaging.com) The important detail is not that the model is “smart.” The important detail is that Butterfly said it trained the tool on more than 21 million ultrasound images for one specific output, then compared its estimates with biometry-based assessments by sonographers. (diagnosticimaging.com) The same pattern shows up in cancer diagnostics. Clinical Lab Products reported on April 10 that Ataraxis Breast NEO uses a core needle biopsy pathology slide taken at diagnosis to predict the likelihood of pathologic complete response after neoadjuvant therapy in early-stage breast cancer. (clpmag.com) Neoadjuvant therapy means treatment given before surgery, usually to shrink a tumor first. A pathologic complete response means no invasive cancer is found in tissue removed after that treatment, so predicting it earlier could change which regimen a clinician chooses before the first infusion starts. (clpmag.com) While companies are narrowing the task, medical schools and hospitals are still arguing about the rules. Isaac Kohane wrote this week that trainees are being pushed into “doublethink,” where they are told publicly not to rely on artificial intelligence while watching senior clinicians use it privately in day-to-day care. (zaklab.org) That gap is dangerous because hidden use is harder to supervise than admitted use. If a resident quietly checks an artificial intelligence tool for a draft note, a differential diagnosis, or a patient explanation, the hospital gets none of the benefits of training, auditing, or clear guardrails. (zaklab.org) So the practical playbook is getting clearer in April 2026. Put clinicians in the design loop, aim models at one well-defined task, test them against a real clinical benchmark, and get regulatory clearance or other validation before asking staff to trust them. (hitconsultant.net) (diagnosticimaging.com) (clpmag.com) (zaklab.org) The health care groups most likely to get burned are the ones shopping for a miracle. The ones most likely to get value are the ones buying a very boring tool that saves one phone call, one measurement, one report, or one treatment decision at exactly the point where a clinician already works. (hitconsultant.net) (diagnosticimaging.com)
