Decagon Launches A/B Testing Suite for AI
What happened
Decagon has announced a new experimentation suite for A/B and multivariate testing of AI agents. The platform is designed for rapid iteration on LLM-powered workflows. It provides centralized infrastructure for measuring the impact of changes on both business metrics and user experience outcomes.
Why it matters
- Decagon was founded in 2023 by CEO Jesse Zhang, a Harvard computer science graduate whose previous gaming startup, Lowkey, was acquired by Niantic, and Ashwin Sreenivas, a Stanford masters graduate who previously worked at Palantir. - The company has seen rapid financial growth, achieving a $4.5 billion valuation after its $250 million Series D funding round in January 2026, which was co-led by Coatue and Index Ventures. This valuation tripled in the six months following their Series C round. - A/B testing for LLMs is a complex engineering challenge compared to traditional web testing because of non-deterministic outputs, higher variance requiring larger sample sizes, and the need to balance metrics like accuracy, latency, and token cost. - The new suite enters a competitive MLOps landscape for generative AI, with established players in LLM evaluation and monitoring including Arize AI, LangSmith, and Braintrust. - In the broader AI customer service space, Decagon competes with major incumbents like Salesforce and Zendesk, as well as Bret Taylor's well-funded startup Sierra. - One of the key technical challenges Decagon's platform addresses is "LLM Drift," where underlying models are updated by providers like OpenAI, potentially causing performance regressions that rigorous A/B testing can catch. - The company's primary business is building AI agents for customer support, utilized by companies like Notion, Duolingo, and Substack, with a pricing model based on either a per-conversation or per-resolution fee.
Key numbers
- - Decagon was founded in 2023 by CEO Jesse Zhang, a Harvard computer science graduate whose previous gaming startup, Lowkey, was acquired by Niantic, and Ashwin Sreenivas, a Stanford masters graduate who previously worked at Palantir.
- The company has seen rapid financial growth, achieving a $4.5 billion valuation after its $250 million Series D funding round in January 2026, which was co-led by Coatue and Index Ventures.
Quick answers
What happened in Decagon Launches A/B Testing Suite for AI?
Decagon has announced a new experimentation suite for A/B and multivariate testing of AI agents. The platform is designed for rapid iteration on LLM-powered workflows. It provides centralized infrastructure for measuring the impact of changes on both business metrics and user experience outcomes.
Why does Decagon Launches A/B Testing Suite for AI matter?
Decagon was founded in 2023 by CEO Jesse Zhang, a Harvard computer science graduate whose previous gaming startup, Lowkey, was acquired by Niantic, and Ashwin Sreenivas, a Stanford masters graduate who previously worked at Palantir. The company has seen rapid financial growth, achieving a $4.5 billion valuation after its $250 million Series D funding round in January 2026, which was co-led by Coatue and Index Ventures. This valuation tripled in the six months following their Series C round. A/B testing for LLMs is a complex engineering challenge compared to traditional web testing because of non-deterministic outputs, higher variance requiring larger sample sizes, and the need to balance metrics like accuracy, latency, and token cost. The new suite enters a competitive MLOps landscape for generative AI, with established players in LLM evaluation and monitoring including Arize AI, LangSmith, and Braintrust. In the broader AI customer service space, Decagon competes with major incumbents like Salesforce and Zendesk, as well as Bret Taylor's well-funded startup Sierra. One of the key technical challenges Decagon's platform addresses is "LLM Drift," where underlying models are updated by providers like OpenAI, potentially causing performance regressions that rigorous A/B testing can catch. The company's primary business is building AI agents for customer support, utilized by companies like Notion, Duolingo, and Substack, with a pricing model based on either a per-conversation or per-resolution fee.
