Shipping with 9 AI Agents: A Postmortem
A development team shared a candid postmortem on building a real product using nine specialized AI agents as their dev team. While the agents completed 52 tasks and merged 56 pull requests, the team highlighted major challenges in coordination, context gaps, and the critical need for human oversight—a reality check on the state of agentic workflows.
The experiment's core challenge wasn't just coding, but orchestrating a team of distinct AI agents. This mirrors a growing trend in complex AI systems, where multiple specialized agents collaborate to solve problems that a single, monolithic AI cannot. The coordination of these agents, including task allocation and communication, introduces significant overhead and new failure modes if not managed carefully. A primary hurdle identified was the "context gap"—the discrepancy between the implicit knowledge a human developer has and what an AI agent can understand from its inputs. AI agents often lack awareness of the broader codebase architecture, unwritten team conventions, and the history of technical decisions. This can lead to AI-generated code that, while syntactically correct, introduces bugs, violates established patterns, or creates technical debt. The need for human oversight in these agentic workflows is not just a temporary measure but a fundamental requirement for safety and quality. This oversight is crucial for aligning AI decisions with ethical standards, ensuring accountability, and intervening to correct errors. Studies have shown that without rigorous human review, AI-generated code can have a higher rate of logic and correctness errors. This postmortem serves as a practical case study on the limitations of current AI software engineers like Devin, which, despite impressive autonomous capabilities, often struggle with the ambiguity and complexity of real-world projects. While Devin can break down complex tasks and even attempt to fix CI failures, its performance on benchmarks like SWE-bench shows a success rate that still lags behind the needs of production environments. For indie hackers and bootstrappers, the rise of AI agents presents both an opportunity and a challenge. These tools can accelerate development by automating coding tasks and optimizing user experiences, allowing small teams to compete with larger companies. However, the most successful implementations will likely involve a hybrid approach, combining AI for specific tasks with the strategic and creative oversight that only human developers can provide. The evolution of AI in development is shifting from AI-assisted humans to human-assisted AI. This new paradigm requires developers to become adept at "context engineering"—structuring and providing the necessary information for AI agents to perform effectively. The focus is less on writing individual lines of code and more on designing and managing the systems in which AI agents operate.
