India's Supreme Court Warns Against AI in Law
Judges in India citing fake, AI-generated case law is not just an error but amounts to "misconduct," the country's Supreme Court has warned. The caution highlights a global concern over the misuse of generative AI in high-stakes professional fields where human judgment and verified facts are critical.
The specific warning from India's Supreme Court arose from a case where a trial court in Andhra Pradesh cited non-existent, AI-generated judgments in a property dispute. A bench of Justices P.S. Narasimha and Alok Aradhe declared on February 27 that relying on such fake precedents is not merely an error but constitutes "misconduct" for which legal consequences will follow. The Supreme Court has escalated the issue, seeking responses from the Attorney General of India, the Solicitor General, and the Bar Council of India to deliberate on the consequences and accountability for using such AI-generated misinformation. Senior Advocate Shyam Divan has been appointed as an amicus curiae, a friend of the court, to provide assistance in this matter. This incident is not isolated. Chief Justice of India, D.Y. Chandrachud, has repeatedly spoken about the inevitable integration of AI in the judiciary while cautioning against its pitfalls. He has highlighted the potential for AI to amplify discrimination and the need to ensure technology aligns with human rights and freedoms, viewing AI as a "double-edged sword." While flagging the risks, the Indian judiciary has been cautiously exploring AI's potential to tackle its massive case backlog, which exceeds 46 million cases. The Supreme Court has already implemented AI tools like SUPACE (Supreme Court Portal for Assistance in Court's Efficiency) for research and has been using AI for live transcription of proceedings into regional languages. The challenge of "AI hallucinations" in legal settings is a global phenomenon, with courts in the United States, United Kingdom, and Canada also sanctioning lawyers for submitting briefs with fictitious case citations generated by tools like ChatGPT. These events have prompted legal bodies worldwide to issue guidelines on the responsible use of AI. In response to these emerging challenges, various High Courts in India, such as Kerala's, are proactively establishing their own policies for the use of AI. These guidelines typically permit AI for administrative tasks and legal research but strictly prohibit its use for judicial reasoning or final decision-making, mandating human oversight at all stages.
