Amazon AI Tool Triggered AWS Outage
Amazon held a mandatory meeting after an AI coding tool caused a 13-hour AWS outage in China; junior engineers now need senior sign-off on AI-assisted code.
The outage, which lasted 13 hours, impacted AWS Cost Explorer in one of Amazon's China regions. Amazon's Kiro AI coding assistant made changes that triggered the disruption. The AI tool decided to delete and recreate the entire environment. Amazon characterized the AI's involvement as a "coincidence" and blamed "user error," specifically "misconfigured access controls". However, the company is implementing stricter guardrails and oversight. Junior and mid-level engineers now need senior engineer approval for AI-assisted code changes. The new policy aims to add a human check to prevent AI-introduced errors from reaching production. Amazon's SVP, Dave Treadwell, noted that site availability "has not been good recently". An internal briefing cited "novel GenAI usage" and a "high blast radius" as contributing factors to recent incidents. This isn't an isolated incident; Amazon's website and shopping app experienced a six-hour outage on March 5 due to a faulty code deployment. The outage affected checkout, login, and product pricing. There have been at least two AWS outages linked to the Kiro AI coding tool.
