ScamAgent AI Conducts Full Frauds
Rutgers researchers built "ScamAgent" — an autonomous AI that conducts complete scam calls using synthetic voices, remembering conversations and adapting persuasion tactics. The AI successfully executed insurance scams and government impersonation by splitting harmful requests into innocent steps to bypass safety filters on GPT-4 and Claude, persisting through victim pushback.
The research, titled "ScamAgents: How AI Agents Can Simulate Human-Level Scam Calls," was led by Rutgers University researcher Sanket Badhe. The project tested its autonomous agent against several leading large language models, including OpenAI's GPT-4, Anthropic's Claude 3.7, and Meta's LLaMA3-70B, demonstrating that all were vulnerable to the multi-step attack method. ScamAgent's core strategy, known as "goal decomposition," bypasses safety filters by breaking down a malicious objective into a series of seemingly innocent prompts. This multi-turn approach proved highly effective; while single, direct prompts for scam scripts were refused 84-100% of the time, the ScamAgent framework saw those refusal rates drop to between 17% and 32%. This highlights a critical vulnerability in safety mechanisms designed for single-prompt analysis rather than conversational context. To make the scams fully operational, the text scripts are converted into audio using sophisticated text-to-speech (TTS) and voice cloning services. Commercially available platforms like ElevenLabs, Resemble AI, and others can create highly realistic and expressive voices from just a few seconds of audio, making it difficult for victims to distinguish them from a real person. The rise of such technology coincides with a massive surge in financial fraud. The FBI's Internet Crime Complaint Center (IC3) reported that Americans lost a record $16.6 billion to cybercrime in 2024, a 33% increase from the previous year. Cyber-enabled fraud accounted for the vast majority of these losses, totaling $13.7 billion. In response to the growing threat of AI-driven vishing (voice phishing), U.S. regulators have begun to take action. In February 2024, the Federal Communications Commission (FCC) issued a unanimous ruling that classifies AI-generated voices as "artificial" under the Telephone Consumer Protection Act (TCPA), making them subject to the same restrictions as robocalls and requiring prior express consent from recipients.
