Incident: Search answers built on jokes and forum posts

Module 8 · Incident file · About 35 minutes

What happened

At its I/O conference in May 2024, Google announced that AI Overviews, model-written summaries at the top of search results, would be available to everyone in the United States. Within days, screenshots of strange answers spread. The Register reported (Matthew Connatser, "Google finally addresses those bizarre AI search results", 31 May 2024) that advice to put glue on pizza came from a user "joking around on Reddit", and that a recommendation to eat rocks was "based on an article from The Onion", the satirical newspaper.

On 30 May 2024 Elizabeth Reid, Google's vice president for Search, published "AI Overviews: About last week". It is unusual: a vendor explaining a RAG product's failure in its own words. AI Overviews, she wrote, are "integrated with our core web ranking systems" and "built to only show information that is backed up by top web results". They "generally don't 'hallucinate'". The errors had other causes. On rocks: "Prior to these screenshots going viral, practically no one asked Google that question", and satirical content on the topic had been "republished on a geological software provider's website", which the AI Overview linked to. On glue: forums give first-hand information but also "less-than-helpful advice, like using glue to get cheese to stick to pizza."

The reports put no money figure on it; the cost was public, and Google reported "more than a dozen technical improvements", and the post also says many circulating screenshots were faked. This file uses only the two cases Google itself explained.