AI scaling hits limits
- Datadog’s new report says companies are hitting operational complexity, not model accuracy, as the main barrier to scaling AI reliably. - The report, published April 21, frames integration, monitoring and engineering overheads as the core obstacles. - ASEAN manufacturers report similar problems, prompting A*Star and Microsoft partnerships to tackle integration and productivity challenges (globenewswire.com) (businesstimes.com.sg).
Companies are finding that getting artificial intelligence into production is harder than getting the model to answer a prompt. Datadog said April 21 that operational complexity is now the main barrier to scaling AI reliably. (financialcontent.com) Datadog’s “State of AI Engineering 2026” report drew on production data from thousands of organizations. It found 69% now use three or more models, and about 5% of AI model requests fail in production. (financialcontent.com) Nearly 60% of those failures were tied to capacity limits, according to the report, rather than the model simply giving a bad answer. Datadog also said agent framework adoption doubled year over year, adding more moving parts to production systems. (financialcontent.com) In plain terms, the problem is shifting from “can the model do this task?” to “can the company keep the whole system running?” That system includes model routing, retries, token usage, infrastructure capacity and monitoring across the application stack. (financialcontent.com) Datadog said median-use teams more than doubled the number of tokens sent per request, while heavy users quadrupled it. Larger prompts and multi-step agent workflows can raise costs, slow responses and create more points of failure if capacity is tight. (financialcontent.com) The same pattern is showing up in factories in Southeast Asia. At Hannover Messe on April 20, panelists said manufacturers in the Association of Southeast Asian Nations are struggling to scale AI not because they lack awareness, but because they face execution problems, return-on-investment concerns and difficulty integrating new tools with legacy systems. (businesstimes.com.sg) Singapore Economic Development Board executive vice-president Cindy Koh said manufacturing accounts for about 22% of Association of Southeast Asian Nations gross domestic product. She said foreign direct investment into the region rose about 8% in 2024 to US$226 billion, with manufacturing taking the second-largest share at US$44 billion. (businesstimes.com.sg) Singapore’s Agency for Science, Technology and Research, known as A*STAR, and Microsoft signed a memorandum of understanding on April 20 at Hannover Messe to explore AI tools for manufacturing. The partners said they will work on an agentic AI platform, deployment infrastructure and broader industry adoption. (edb.gov.sg) A*STAR said manufacturers often lack combined expertise in AI and manufacturing, struggle with data availability and standardization, and worry about deploying AI reliably in industrial settings. It said the platform under discussion would stay platform-agnostic, so companies could use it within existing information-technology systems. (edb.gov.sg) Datadog’s report also showed the supplier mix is widening as companies juggle more systems. OpenAI held a 63% provider share in the report, while Google Gemini and Anthropic Claude each posted sizable gains, a setup that can improve flexibility but also increase integration and monitoring work. (financialcontent.com) The new bottleneck is not whether AI can produce an answer on a benchmark. It is whether companies can connect models, data, software and hardware tightly enough that the answer arrives on time and keeps arriving. (financialcontent.com)
