The New York Times recently reported that Google's AI Overviews have an accuracy rate of about 90%. This data is based on more than 5 trillion searches per year, meaning that over 57 million incorrect answers could be generated every hour, nearly 1 million pieces of incorrect information per minute.
Startup company Oumi evaluated Google Search and analyzed 4,326 search results using the SimpleQA benchmark. The results showed that Google's Gemini 2 had an accuracy rate of 85% in October last year, while the accuracy of Gemini 3 increased to 91% by February this year. However, Oumi's evaluation method mainly relied on AI tools, which may lead to biased data. In addition, Google may generate different overviews for the same search query, increasing the uncertainty of the results.
Specifically, the proportion of AI overviews that do not match the original information sources has risen from 37% with Gemini 2 to 56% with Gemini 3. This means that users often find links that contradict the overview or accurate overviews that cite incorrect information. A journalist even published a fake blog, and Google included related content in its overview the next day, showing that AI overviews are vulnerable to manipulation.
Additionally, user Stephen Pavalescu searched for news about wrestler Hulk Hogan's death, and the AI overview clearly stated "There is no credible report that Hogan has died," but the article title below the overview was "The Mystery of Hogan's Death Deepens." This contradictory phenomenon has raised doubts among users about the reliability of AI-generated content.
In response to these issues, a Google spokesperson questioned Oumi's testing methods, arguing that the evaluation did not accurately reflect actual search behavior.
Key Points:
🔍 The accuracy of AI overviews is approximately 90%, potentially generating 57 million incorrect answers per hour.
📈 The accuracy of Gemini 3 has increased to 91%, but the proportion of overviews inconsistent with the original information has risen to 56%.
🤖 AI overviews are susceptible to false information, leading users to question their reliability.
