Therapy chats and privacy gaps
Several reports flagged that conversational AI used in mental‑health apps can be unprotected by HIPAA and that some platforms may be compelled to hand over chat data, prompting at least one U.S. state to ban AI therapy with fines up to $10,000 per violation (x.com). A separate Swiss case highlighted patient audio being routed to U.S. AI services without proper security or consent, raising cross‑border re‑identification and compliance concerns (x.com).
A therapy chat with a bot can fall outside medical privacy law, and one state has already banned artificial intelligence from providing therapy. (ilga.gov) Illinois enacted House Bill 1806 on August 4, 2025, creating the Wellness and Oversight for Psychological Resources Act. The law says therapy offered to the public in Illinois must be conducted by a licensed professional and allows civil penalties of up to $10,000 per violation. (ilga.gov) Governor JB Pritzker’s administration said the law bars anyone from using artificial intelligence for “mental health and therapeutic decision-making” while still allowing administrative support for licensed clinicians. The Illinois Department of Financial and Professional Regulation said the measure took effect immediately when Pritzker signed it on Friday, August 4, 2025. (idfpr.illinois.gov) The privacy gap starts with who is covered. The Health Insurance Portability and Accountability Act applies to covered health providers, health plans, and their business associates, not to every wellness app or chatbot that collects health information. (hhs.gov) Federal regulators updated the Federal Trade Commission’s Health Breach Notification Rule on July 29, 2024, to clarify that many health app developers are outside the Health Insurance Portability and Accountability Act but still must report breaches of unsecured health data. The rule says it covers vendors of personal health records and related entities that are not covered by the Health Insurance Portability and Accountability Act. (federalregister.gov) The rule also says it applies to foreign and domestic vendors that maintain information on United States citizens or residents, and it excludes Health Insurance Portability and Accountability Act covered entities and business associates acting in that role. That means the legal regime can change depending on whether a chatbot is part of a hospital workflow or a stand-alone consumer app. (ecfr.gov) The second gap is legal privilege. In a July 2025 podcast interview, OpenAI chief executive Sam Altman said there is no doctor-patient or attorney-client style confidentiality for conversations with ChatGPT and said OpenAI could be required to produce those chats in litigation. (techcrunch.com) That warning landed as a federal court in New York ordered OpenAI on May 13, 2025, to preserve and segregate output log data that otherwise would have been deleted in the copyright case against the company. Bloomberg Law reported the dispute has become a test of how courts balance discovery demands against the privacy interests of hundreds of millions of users. (cdn.arstechnica.net; news.bloomberglaw.com) A separate set of risks appears when clinics use “ambient” artificial intelligence scribes, which record a visit and turn it into a draft note. The American Bar Association said recent lawsuits in California and Illinois allege health systems used those tools without informed consent when patient audio was transmitted to third-party vendors. (americanbar.org) Swiss regulators have taken the same basic position on transparency and control. The Federal Data Protection and Information Commissioner said on September 23, 2025, that Switzerland’s Federal Act on Data Protection applies directly to artificial intelligence systems and requires providers to explain purpose, function, and data sources, while giving users the right to know when they are interacting with a machine. (edoeb.admin.ch) Cross-border transfers add another compliance test. Switzerland has allowed transfers to certified United States companies under the Swiss-United States Data Privacy Framework since September 15, 2024, but transfers outside that framework still require added safeguards or the data subject’s consent under the Federal Act on Data Protection. (swlegal.com) The practical line is simple: a chat that feels like therapy is not automatically protected like therapy, and a recording that sounds like routine note-taking can still trigger consent, security, and cross-border data rules. Illinois has now written that line into state law, and other regulators are moving on the same terrain. (ilga.gov; edoeb.admin.ch)
