Expert Recommends Security Limits for AI Agents
Guidance on securing AI agents suggests treating them like a new hire without credentials. Recommended security measures include isolating the agent on separate hardware, providing read-only data access, and withholding passwords. The advice also stresses the importance of admin controls to prevent agents from taking destructive actions, such as deleting an entire email inbox.
- The Open Web Application Security Project (OWASP) has identified "Excessive Agency" as a top 10 security risk for Large Language Model (LLM) applications. This occurs when an agent is granted more permissions than necessary, which can be exploited through prompt injection to access sensitive data or execute unauthorized actions. - Traditional security tools like static Role-Based Access Control (RBAC) are often inadequate for AI agents because agents' behaviors are dynamic and can't be easily predicted. Security for AI agents is shifting towards an identity-first approach, treating each agent as a non-human identity that requires continuous verification and dynamic authorization. - A significant blind spot for enterprise security is the rise of "Shadow AI," where business units deploy AI agents without direct IT or security oversight. This often happens through OAuth grants in SaaS applications, IDE extensions, and other integrations that security teams may not be actively monitoring. - Frameworks like the NIST AI Risk Management Framework (RMF) are being adapted to provide guidance on governing AI agents. The framework emphasizes creating a culture of risk management, mapping out the context of AI systems, and continuously measuring and monitoring for emergent risks. - Real-world security incidents have demonstrated the risks of autonomous agents, such as a Chevrolet dealership's AI chatbot being manipulated through prompt injection to agree to sell a car for $1. Other incidents include the accidental exposure of 38 terabytes of Microsoft's private data through a misconfigured token. - A key vulnerability in AI agents is the "confused deputy problem," where an agent with elevated permissions is tricked by a malicious input to misuse its authority. This is why security best practices emphasize external and deterministic validation of an agent's actions, rather than relying on the LLM to validate its own behavior. - To mitigate risks, a defense-in-depth approach is recommended, which includes limiting an agent's tool scope to read-only by default and enforcing parameter restrictions at the application level, not within the model's reasoning process. Additionally, each agent should have its own separate identity and credentials, rather than sharing keys or inheriting broad roles. - The supply chains for AI agents, including the frameworks, plugins, and embedding models they rely on, are potential attack vectors. Vulnerabilities in these third-party components can be inherited by the agent, making supply chain security a critical aspect of AI agent deployment.
