Child-AI Safety Alarm
- Investigative reporting says AI has emboldened child predators and investigators are struggling to respond. - California advocacy groups are pushing SB 1119 and AB 2023 to tighten chatbots' protections for children. - New research shows adversarial prompting can bypass safeguards, increasing pressure to build product-level safety beyond model filters. ( )
Child predators are using artificial intelligence to make abuse material faster, cheaper, and harder for police to trace, and California lawmakers are moving to tighten chatbot rules for minors. (bloomberg.com, transparencycoalition.ai) Bloomberg reported on April 23 that easier-to-use AI tools have already increased abusive imagery and made online child sexual abuse cases harder for law enforcement to investigate. California’s latest response is a pair of companion bills, Senate Bill 1119 and Assembly Bill 2023, introduced February 17, 2026. (bloomberg.com, legiscan.com, legiscan.com) SB 1119 cleared its first policy committee on April 20 in a 7-0 vote and was re-referred to the Senate Judiciary Committee. The Assembly companion, AB 2023, remains in committee after amendments in March. (legiscan.com, legiscan.com) The bills would require chatbot operators to verify a user’s age, run annual child-safety risk assessments, publish a child-safety policy, and submit to independent audits whose reports go to the California attorney general. They would also let public prosecutors and children harmed by violations bring civil actions. (transparencycoalition.ai, legiscan.com) California is not starting from zero. Governor Gavin Newsom signed SB 243 in 2025, requiring companion chatbots to disclose that they are artificial, warn minors, and maintain protocols to prevent suicidal-ideation and self-harm content. (perkinscoie.com, legiscan.com) The new push reflects a broader problem with how chatbot safety works. Many systems rely on prompt guards, which are lightweight filters that scan what a user types before the main model answers, like a metal detector at the door rather than a lock on the vault. (arxiv.org) A recent paper found those guards can be bypassed when an attacker hides a jailbreak inside text the main model can decode but the lighter filter cannot. The authors said the method worked against production chat interfaces including Google Gemini 2.5 Flash and Pro, DeepSeek Chat, Grok 3, and Mistral Le Chat. (arxiv.org) Another 2026 study found that changing style alone can raise failure rates: across 25 frontier and open-weight models, turning harmful requests into verse pushed jailbreak success rates as high as 18 times the prose baseline, with some providers above 90% on the authors’ metric. (arxiv.org) Those results do not prove a chatbot will harm a child in any single conversation, but they do show why lawmakers and child-safety advocates are asking for product-level controls such as age checks, time limits, crisis protocols, and audits instead of relying only on model filters. California sponsors said their March 26 amendments would also add default privacy settings and one-hour session and two-hour daily caps for minors. (arxiv.org, arxiv.org, bauer-kahan.asmdc.org, sd18.senate.ca.gov) The immediate test is whether California can turn those safeguards into enforceable rules before July 1, 2027, the compliance date written into SB 1119 and AB 2023. Investigators are already dealing with the AI version of the threat. (legiscan.com, legiscan.com/CA/text/AB2023/id/3405154, bloomberg.com)
