Enterprise AI settling into three layers
Consulting analysis says enterprise AI platforms are converging on a three-layer architecture that separates model execution, orchestration/control, and governed data access — a framing that clarifies where internal platform teams add the most value. The pattern suggests platform teams should focus on shared control planes (identity, telemetry, policy, routing) rather than trying to replace every vendor runtime. (The Three Layers of an Agentic AI Platform | Bain & Company)
Most companies spent 2023 and 2024 wiring one chatbot to one model at a time. Bain says that setup breaks once an artificial intelligence system has to plan, call tools, check policies, and pull company data across many systems. (bain.com) The new pattern looks less like one app and more like a stack with three jobs. One layer runs the models, one layer coordinates the work, and one layer controls access to trusted data. (bain.com) The bottom layer is the model layer, which is the engine room. Bain places large language models, vector databases, cloud compute, and vendor services here, which means this is where tokens get processed and outputs get generated. (bain.com) The middle layer is the control layer, which acts more like air traffic control than a single app. Bain puts workflow orchestration, memory, observability, guardrails, routing, and human approval loops in this layer because agents do not just answer once and stop. (bain.com) The top layer is the access layer, which decides what the system is allowed to know. Bain says this layer connects governed enterprise data, application programming interfaces, and business tools so an agent can reach customer records or finance systems without turning the whole company into an open file cabinet. (bain.com) That split changes what an internal platform team should build. Bain’s argument is that most companies should not try to replace OpenAI, Anthropic, or a cloud provider’s runtime, because the durable advantage sits in shared policy, telemetry, identity, and routing across all of them. (bain.com) You can already see the vendors moving into those lanes. OpenAI’s enterprise tools include role-based access controls, audit logs, usage controls, and compliance features, which are classic control-plane functions rather than end-user chat features. (openai.com, developers.openai.com, openai.com) Anthropic is doing the same from a different angle. Its current developer and enterprise materials emphasize tool use, retrieval-augmented generation, the Model Context Protocol, managed agents, and admin controls, which all fit the orchestration and governed-access parts of the stack. (anthropic.com, anthropic.com, assets.anthropic.com) That is why the middle layer is becoming the crowded battleground. If one company uses OpenAI for coding help, Anthropic for long documents, and a cloud model for internal search, the control layer is the piece that decides which model gets which job, what data it can see, and what gets logged for audit. (bain.com, developers.openai.com, anthropic.com) The practical takeaway is narrower than “build your own platform” and more specific than “buy one vendor suite.” Build the shared rails for identity, policy, telemetry, approval, and routing once, then let model runtimes and business applications swap in and out above and below them. (bain.com, bain.com) That architecture also explains why so many pilots looked good in demos and stalled in production. Bain’s earlier work says most pilots meet expectations but few create measurable value at scale, and the missing piece is usually not another model upgrade but the plumbing that makes many models, many tools, and many data systems behave like one governed system. (bain.com, bain.com)
