Google’s AI Overviews show accuracy issues

Google’s new search habit is to answer first and link later, and that only works if the answer is right nearly every time. New reporting on outside tests found Google’s Artificial Intelligence Overviews miss the mark on about 9% to 10% of prompts, which sounds small until you put it next to Google’s scale. (arstechnica.com) An Artificial Intelligence Overview is the box at the top of some Google results that writes a short answer by stitching together information from multiple web pages. It is not the old blue-links page with snippets from one site at a time; it is a machine-written summary that tries to act like a finished answer. (blog.google) Google has pushed that box very hard. By October 2024, the company said Artificial Intelligence Overviews were rolling out to more than 100 countries and reaching more than 1 billion monthly users, and by May 2025 Google said the feature had grown to more than 1.5 billion users in 200 countries and territories. (blog.google 1) (blog.google 2) The reason a 10% error rate turns into a giant number is that Google is giant. Google said in 2025 that people make more than 5 trillion searches on Google each year, so even a small failure rate can become tens of millions of bad answers in an hour if the feature appears often enough. (blog.google) (popsci.com) The underlying problem is simple: these systems do two jobs at once. First they have to fetch the right pages from the web, and then they have to rewrite those pages into one clean paragraph without dropping context, mixing up sources, or inventing a detail that was never there. (blog.google) (arstechnica.com) That second step is where polished nonsense can slip in. A normal search result can send you to a bad page, but an Artificial Intelligence Overview can combine several decent pages into one confident sentence that none of those pages actually said. (arstechnica.com) (popsci.com) Google has dealt with this before. In May 2024, after users shared bizarre answers about things like putting glue on pizza, Google said many of the viral examples were uncommon queries, satirical content, or manipulated prompts, and it announced changes meant to limit when Artificial Intelligence Overviews would appear. (blog.google) (popsci.com) But reducing the weirdest failures is not the same as solving the quiet ones. The new concern is not only cartoonishly wrong answers; it is ordinary-looking mistakes in health, history, shopping, travel, or finance queries that most people will never double-check because the box sits above the rest of the page with Google’s branding around it. (popsci.com) (technology.org) There is another shift underneath this one: when the summary appears, people click links less. Search Engine Land reported in 2025 that Artificial Intelligence Overviews were cutting click-through rates to regular search listings, and Pew Research Center later found users were less likely to click outside links when an Artificial Intelligence summary showed up. (searchengineland.com) (pewresearch.org) That means Google is taking on more responsibility than a traditional search engine used to carry. If the page sends fewer people to the original sources, then the summary itself is no longer a shortcut to the web; for many users it is the web page they trust. (pewresearch.org) (blog.google) The hard part for Google is that speed and coverage are easy to scale, but caution is expensive. A safer system would decline more questions, show fewer summaries, or fall back to ordinary links more often, and every one of those moves makes the product feel less magical even if it makes it more honest. (blog.google) (arstechnica.com) So the real story is not that Google’s search box sometimes says something dumb. It is that a feature used by more than 1.5 billion people is trying to compress the open web into one paragraph at industrial scale, and even a single-digit miss rate becomes a public-information problem when the machine is answering first. (blog.google) (popsci.com)