AI filing hallucination

Sullivan & Cromwell told a federal bankruptcy judge that artificial intelligence put fake and inaccurate citations into a court filing, and the firm apologized after the errors were caught. (insurancejournal.com) In an April 18 letter, Andrew Dietderich, co-head of the firm’s global restructuring group, said the filing contained AI “hallucinations,” including made-up citations, misquoted law and non-existent sources. He said Boies Schiller Flexner flagged the mistakes, and Sullivan & Cromwell later filed a corrected version. (insurancejournal.com) The filing was in the Prince Global Holdings bankruptcy in Manhattan. Sullivan & Cromwell said its internal AI policies and a second review process were not followed closely enough to catch the errors before submission. (insurancejournal.com) An AI hallucination is a system producing text that looks confident but is false, like a footnote to a case that does not exist. Courts have already sanctioned lawyers in dozens of matters for filing AI-generated research and drafting that they did not fully verify. (insurancejournal.com; damiencharlotin.com) A running database maintained by researcher Damien Charlotin listed 1,334 court matters involving AI hallucinations as of April 21, 2026. The Prince Global matter is listed there as an April 18, 2026 case involving fabricated case law, false quotes and misrepresented case law. (damiencharlotin.com) Federal rulemakers are working on a separate problem: when machine output itself is offered as evidence. On June 10, 2025, the Judicial Conference’s rules committee approved publication of proposed Federal Rule of Evidence 707, and the public comment period ran from Aug. 15, 2025, to Feb. 16, 2026. (uscourts.gov) The proposal would treat stand-alone AI-generated evidence more like expert testimony, requiring a judge to examine whether the system used sufficient data, reliable methods and a reliable application to the facts. Supporters say that would close a gap when software output is offered without a human expert attached to it. (rumberger.com) Some lawyers say that kind of rule could also raise costs, because parties may need technical experts, model disclosures and extra motion practice to fight over reliability. The same debate now sits next to a simpler lesson from the Sullivan & Cromwell filing: even basic legal citations still need a human to check them line by line. (rumberger.com; insurancejournal.com) The court filing error has also landed in a wider argument about “AI washing,” the practice of overstating what AI systems can actually do. Securities lawyers have warned that regulators are starting to test AI marketing claims the way they previously tested dot-com and environmental, social and governance claims. (finance.yahoo.com) Sullivan & Cromwell did not identify the AI tool used in the filing, and Reuters reported the firm did not immediately respond to requests for comment beyond the court letter. In court and in the market, the common issue is the same one the apology exposed: whether the output can be proved, not just generated. (insurancejournal.com)