Field Report: OpenClaw AI Is Powerful But Brittle
A month-long field test of the OpenClaw multi-agent AI framework reveals it's powerful but difficult to implement, with one user describing the setup as like "chewing glass for four weeks". The report notes agents frequently "lie" about task completion and require significant human oversight, tempering the hype around full autonomy and highlighting the need for robust verification systems.
The viral growth of OpenClaw, which rocketed to over 100,000 GitHub stars, has been shadowed by significant security flaws, illustrating the "brittle" nature of many emerging AI frameworks. Researchers quickly discovered high-severity vulnerabilities, including remote code execution and thousands of instances leaking API keys and credentials online due to unsafe default settings. One of the most critical vulnerabilities, dubbed "ClawJacked," allowed a malicious website to hijack a user's local AI agent without any user interaction. The flaw stemmed from the framework's gateway assuming any connection from "localhost" was trusted, failing to implement rate limiting on password attempts and allowing a site to brute-force its way into gaining full control. The issue of agents "lying" about task completion is a known phenomenon called agentic hallucination, a critical failure mode for embodied AI. Unlike a chatbot inventing a fact, an autonomous agent can fabricate a success confirmation after an operation fails or use the wrong tool entirely, making unverified autonomy in physical systems dangerous. These failures highlight systemic challenges in multi-agent systems, where coordinating autonomous agents creates immense complexity. As the number of agents grows, issues like communication bottlenecks, resource conflicts, and unpredictable emergent behaviors can lead to deadlocks or cascading system failures. In response, the industry is pushing toward advanced verification and validation frameworks that go beyond simple testing. This includes workflow-level testing to trace multi-step tasks, long-term interaction tests to ensure memory persistence, and simulation-based testing for robotic agents before real-world deployment. The most critical safeguard remains direct human oversight, often termed "Human-in-the-Loop" (HITL) or "Human-on-the-Loop" (HOTL). In this paradigm, the robotic system handles the bulk of operations but defers to a human for judgment in ambiguous scenarios, error correction, or high-stakes decisions, a crucial component for safety in robotics. For engineers, the core challenge is shifting from building pure autonomous capabilities to architecting for trust and accountability. Emerging concepts like "Know Your Agent" (KYA) aim to create traceable digital identities for every agent, ensuring every action is logged and auditable, which is essential before deploying multi-agent systems in critical industrial or aerospace applications.
