Menlo: RAG powers 51% of enterprise AI
- Menlo Ventures’ November 20, 2024 enterprise AI report says retrieval-augmented generation, or RAG, became the most common production architecture inside companies. - The headline number is 51% adoption for RAG, up from 31% a year earlier, while only 9% of production models were fine-tuned. - That matters because enterprises are choosing faster, data-connected systems over custom-trained models as AI moves from pilots into core workflows.
Enterprise AI has spent two years arguing about the glamorous part — model quality, model size, model rivalry. But inside actual companies, the winning pattern has been much less dramatic. Menlo Ventures’ November 20, 2024 enterprise report says RAG, not fine-tuning, is now the dominant way production AI systems get built. That matters because it tells you what companies really value when they stop demoing and start deploying: fresh data, lower friction, and something teams can ship without retraining a model every time the business changes. (menlovc.com) ### What is RAG, in plain English? RAG — retrieval-augmented generation — means the model does not rely only on what was baked into its weights during training. It fetches relevant documents at runtime, then answers with that material in view. Basically, it is an LLM with a live notebook open beside it. That notebook can be a company wiki, product docs, support tickets, contracts, or an internal knowledge base. (menlovc.com) ### Why are enterprises leaning that way? Because enterprise knowledge changes constantly. Policies update. Prices move. Inventory shifts. New contracts land. If your AI system needs current answers, retraining the model every time is slow and expensive. RAG lets teams keep the base model mostly intact and swap in better retrieval, cleaner data pipelines, and tighter permissions instead. That is a much more natural fit for company information systems. (menlovc.com) ### What did Menlo actually measure? Menlo’s 2024 State of Generative AI in the Enterprise report surveyed 600 U.S. enterprise leaders and framed 2024 as the year AI moved “from pilots to production.” In that dataset, RAG reached 51% adoption in production, up from 31% the year before. Fine-tuning showed up in only 9% of production models. The gap is the story — the defaul(menlovc.com).” (menlovc.com) ### Why is fine-tuning losing here? Fine-tuning is still useful — but mostly when you need a model to behave in a very specific way, learn a narrow style, or perform a repeated task with stable patterns. The catch is that many enterprise problems are not stable. They are messy knowledge problems. A legal assistant needs the latest policy memo. A support bot needs the newes(menlovc.com)r because the knowledge lives outside the model. (menlovc.com) ### Does this mean fine-tuning is dead? No — just demoted. Menlo’s own framing in later architecture pieces is that enterprise systems have evolved from prompting to RAG and then toward agents, not that one technique erased all others. Fine-tuning still matters for specialized behavior, but it is no longer the center of gravity for most production deployments. Turns out th(menlovc.com)ation into the answer.” (menlovc.com) ### Why does this feel like a platform shift? Because it changes where value accrues. If RAG is the default architecture, then the hard problems move toward data ingestion, indexing, permissions, evaluation, orchestration, and application design. Menlo’s 2024 report says enterprise AI spending jumped to $13.8 billion from $2.3 billion in 2023, with the app(menlovc.com)like — less obsession with the raw model, more spending on the stack around it. (menlovc.com) ### So what should readers take from the 51% number? It is less a victory lap for one acronym than a reality check about enterprise AI. Companies are telling us, with their architecture choices, that usefulness beats purity. The systems winning in production are the ones that can see the latest company knowledge and answer with it safely. For now, that makes RAG the workhorse. (menlovc.com)
