AI Trust Problems Hit Big Institutions

A Wall Street law firm, a bankrupt AI company, and Senate Democrats have all put the same problem in view this month: institutions are still struggling to trust AI outputs. (insurancejournal.com) (justice.gov) (banking.senate.gov) On April 18, Sullivan & Cromwell told Chief Judge Martin Glenn in Manhattan bankruptcy court that a filing in the Prince Global Holdings case contained inaccurate citations and other errors generated by artificial intelligence. Andrew Dietderich, co-head of the firm’s global restructuring group, said the mistakes included invented citations, misquoted law, and nonexistent legal sources. (insurancejournal.com) Dietderich said Boies Schiller Flexner spotted the errors, the firm apologized, and a corrected filing was submitted. Sullivan & Cromwell also told the court that its internal AI policies and a secondary review process were not followed in that filing. (insurancejournal.com) The basic failure is simple: generative artificial intelligence predicts plausible words and citations, but it does not independently verify that a case, quote, or source is real. In court work, that turns a drafting shortcut into a filing risk, because lawyers remain responsible for every citation they submit. (insurancejournal.com) Federal judges have already sanctioned lawyers in dozens of cases for using AI in legal research and drafting without fully checking the results. The Sullivan & Cromwell incident stands out because the firm has more than 900 lawyers and is one of the country’s best-known corporate law firms. (insurancejournal.com) A second April case moved the trust problem from bad output to alleged fraud. On April 17, federal prosecutors in Brooklyn unsealed a 10-count indictment against iLearningEngines founder and former chief executive Puthugramam “Harish” Chidambaran and former chief financial officer Sayyed Farhan Ali “Farhan” Naqvi. (justice.gov) The Justice Department said the two men falsely inflated iLearningEngines’ revenue by hundreds of millions of dollars through sham contracts involving entities they controlled and other associates. Prosecutors said the company used materially false statements about financial performance to attract retail and institutional investors and obtain financing. (justice.gov) U.S. Attorney Joseph Nocella Jr. said the defendants “exploited investor excitement over the AI boom,” while the Federal Bureau of Investigation arrested Chidambaran in Maryland and Naqvi in California on April 17. The charges include securities fraud, wire fraud, and operating a continuing financial crimes enterprise. (justice.gov) In Washington, Senator Elizabeth Warren and three other Democratic senators pressed the Financial Stability Oversight Council on January 22 to investigate what they called an AI debt bubble. Their letter said more than $1 trillion is projected to be poured into AI infrastructure buildouts and warned that losses could spread through banks, insurers, pensions, real estate investment trusts, and retail investors. (banking.senate.gov) The letter focused on financing, not chatbot mistakes, but it described the same institutional question in a different form: whether claims around AI are being checked before money, legal filings, or risk models move on them. Financial Stability Oversight Council was created after the 2008 crisis to spot threats that jump from one firm to the wider system. (banking.senate.gov) Sullivan & Cromwell said its AI policies were not followed; prosecutors said iLearningEngines executives sold investors a false story; Warren’s letter asked regulators to test whether AI financing is outrunning demand. In each case, the missing control was human verification before institutional decisions were locked in. (insurancejournal.com) (justice.gov) (banking.senate.gov)