Stanford Index: AI Progress Slowing
The 2025 Stanford Index reveals that while AI performance on complex benchmarks continues to improve, the rate of progress is slowing compared to the breakneck pace of 2023-2024. Observers note the next competitive frontier for AI will likely be reliability and vertical integration, rather than just raw benchmark scores.
While the growth in AI benchmark scores may be moderating from the explosive pace of previous years, the frontier of AI capabilities continues to expand into more complex domains. In 2024, new and more challenging benchmarks were introduced to test the limits of advanced AI, and within a year, models showed dramatic improvements. For instance, performance on the GPQA benchmark for graduate-level questions rose by 48.9 percentage points. The competitive landscape at the cutting edge of AI development is becoming increasingly crowded. The performance gap between the top-ranked AI model and the 10th-ranked model has narrowed significantly in the past year, from 11.9% to 5.4%. This suggests that access to top-tier AI capabilities is no longer limited to a handful of leading labs, with high-quality models now available from a growing number of developers. A key trend accelerating this accessibility is the rapid improvement of open-weight models. In early 2024, the leading closed-weight model outperformed its top open-weight counterpart by 8.04% on the Chatbot Arena Leaderboard. By February 2025, that performance gap had shrunk to just 1.70%. This convergence allows for wider innovation as more researchers and developers can build upon powerful base models. The focus of AI development is also shifting from pure benchmark performance to real-world applicability and efficiency. This is exemplified by the rise of "vertical AI," which involves tailoring AI solutions for specific industry needs, from healthcare to finance. This approach prioritizes deep domain knowledge, reliability, and seamless integration into existing workflows over generalized performance. Another dimension of this shift is the increasing sophistication of AI agents. On certain complex tasks with short time horizons, top AI systems can now outperform human experts. However, when given more time, humans still maintain an advantage, highlighting the ongoing evolution of human-AI collaboration. The business world has taken notice of these advancements, with corporate adoption of AI jumping from 55% in 2023 to 78% in 2024. This rapid integration has been accompanied by a surge in AI-related incidents, which saw a 56.4% increase in 2024. In response, U.S. states have dramatically increased the pace of AI-related legislation, passing 131 laws in the past year.
