Google Overviews: errors at scale
A study finds Google’s AI 'Overviews' can produce millions of wrong answers every hour once small error rates scale to search volume. Critics note that even a roughly 90% accuracy rate still implies a huge absolute number of incorrect results, and publishers warn those overviews can bury the links that fund original reporting (analyticsinsight.net) (aol.com).
Google is now accurate enough to be wrong at industrial scale. A new analysis found Google’s artificial intelligence search summaries answered a standard factual test correctly about 91% of the time in February 2026, up from 85% in October 2025, but that still leaves roughly 1 wrong answer in 11. (searchengineland.com) That sounds decent until you multiply it by Google’s size. Google is expected to handle more than 5 trillion searches in 2026, which turns a single-digit error rate into tens of millions of bad answers every hour. (popsci.com) The estimate behind the headline is simple math, not a leak. If Google serves about 5 trillion searches a year, that is about 570 million searches an hour, and a 10% error rate would imply about 57 million wrong results in that hour. (analyticsinsight.net) The test came from Oumi, an artificial intelligence startup, using a benchmark called SimpleQA, which is a set of short factual questions used to check whether a model says true things. The New York Times first reported the Oumi analysis, and several follow-up reports repeated the same basic finding. (techspot.com) Google’s summaries are called AI Overviews, and they sit above the usual blue links on many searches. Google said in May 2024 that the feature was rolling out to everyone in the United States and described it as a “jumping off point” to visit web content. (blog.google) That placement is why publishers are alarmed. If the answer appears at the top of the page, many people never click through to the article that did the reporting, and news groups say those lost clicks cut directly into the money that pays reporters and editors. (aol.com) The complaint is not only that the summaries can be wrong. Personal computer magazine reported that even when an overview is right, the links underneath do not always support the claim the summary makes, which means the citation can look reassuring without actually proving the point. (pcmag.com) This problem has been visible since the launch. In May 2024, Google acknowledged “odd, inaccurate or unhelpful” answers after users shared examples like bizarre food and health advice, and the company said it had added guardrails and limited some triggers. (blog.google) The newer data says those fixes improved the hit rate, but not enough to erase the scale problem. Moving from 85% to 91% accuracy sounds like progress in a lab, yet at Google volume it still means hundreds of thousands of mistakes each minute. (marketing-now.co.uk) That is the real shift in search. The old Google mostly risked ranking the wrong page too high, while the new Google increasingly risks writing the wrong answer itself and placing it where the original source used to earn the click. (techcrunch.com)
