Three layers for agentic AI
Bain says legacy enterprise AI platforms are a poor fit for agentic systems and proposes three core layers—orchestration, observability, and governed data access—to reframe platform architecture for agents. The note argues this framing shifts decisions from ‘how to expose models’ to where control, telemetry and data boundaries should live. (bain.com)
An artificial intelligence agent is software that does multi-step work with tools and data, not just a chatbot that answers one prompt. Bain said in an April 2026 brief that those systems need a different enterprise stack built around orchestration, observability, and governed data access. (bain.com) Bain published the note as part two of a four-part series on agentic architecture, after a March 2026 brief that argued legacy request-response systems break down when agents must plan, call tools, share context, and adapt during runtime. (bain.com) In Bain’s framing, orchestration is the traffic-control layer: it routes work among models, tools, memory, and approval steps. Observability is the tracing layer: it records what the agent did, which tool it called, and where a workflow failed or drifted. (bain.com) The third layer is governed data access, which means agents do not get a flat pass to every system of record. Bain said companies should decide which data an agent can read or write, under what policy, and with what audit trail before broad deployment. (bain.com) That architecture shifts the design question away from exposing a model through an application programming interface and toward controlling execution. Bain said the key choices now sit around runtime control, telemetry, and data boundaries because agentic systems are connected and nondeterministic. (bain.com) Large model vendors have been moving in the same direction. OpenAI said its Responses application programming interface, Agents software development kit, built-in tools, and integrated observability are meant to simplify orchestration and tracing for agent workflows. (openai.com) OpenAI’s developer documentation makes the division of labor explicit: developers use the Agents software development kit when their application owns orchestration, tool execution, approvals, and state. That maps closely to the control layer Bain says enterprises need to define for themselves. (developers.openai.com) Anthropic has made a similar case from the tool side. Its agent guidance says frameworks help with low-level work such as calling models, defining tools, and chaining steps, while its engineering team has pushed for clear tool boundaries and evaluation so agents do not roam across hundreds of functions without control. (anthropic.com 1) (anthropic.com 2) Bain’s broader 2025 technology report said most companies were not ready for agentic artificial intelligence because they still had to modernize systems, data, interoperability, security, and accountability. The new three-layer model turns that readiness problem into a platform blueprint that chief information officers can actually assign to teams. (bain.com) The practical effect is that enterprise debates over agents are moving from “which model should we expose?” to “who controls the workflow, who can inspect it, and what data can it touch?” Bain’s answer is three layers, with governance built in before the agents go live. (bain.com)
