“How many rocks should I eat?”
Somebody typed that into Google in May 2024. At the top of the results sat a new box, an AI Overview, and it had an answer: geologists recommend that humans eat one rock per day.
Someone else asked how to make cheese stick to pizza better. The overview suggested “non-toxic glue.”
By May 24 the BBC was reporting that both answers were being widely mocked on social media. Google had switched AI Overviews on for everyone in the US in mid-May, at its annual developer showcase, after trying it in April on a small number of logged-in users in the UK. Google billed it as something that “can take the legwork out of searching.” By Statcounter’s figures, cited by the BBC, Google handled more than 90% of the world’s searches.
Google had reason to think the feature was ready. Three months earlier, after its Gemini image tool had to be paused, Sundar Pichai had promised staff “robust evals and red-teaming.” AI Overviews, Google later wrote, had been tested extensively before launch, with “robust red-teaming efforts,” samples of typical queries and trials on a share of real search traffic. And they were not built like a chatbot. Google designed them “to only show information that is backed up by top web results,” and so, it said, they generally didn’t make things up.
On the rocks, it hadn’t. Almost nothing on the web seriously considers how many rocks a person should eat, a gap Google calls a data void. The Onion had considered it, not seriously. Its piece, “Geologists Recommend Eating at Least One Small Rock Per Day,” had also been republished on a geological software provider’s website. The overview, in Google’s words, “faithfully linked to one of the only websites that tackled the question.” The glue, Google said, came from “sarcastic or troll-y content from discussion forums.” The BBC reported that some answers appeared to be based on Reddit comments.
At first a Google spokesperson called them “isolated examples.” Then, on May 30, Liz Reid, Google’s head of search, published the company’s own account. Before the screenshots went viral, she wrote, practically no one had asked Google how many rocks to eat. Many screenshots, she added, were fake, including ones suggesting dangerous advice about leaving dogs in cars, smoking while pregnant and depression. “Those AI Overviews never appeared.” Content policy violations, by Google’s count, came up on fewer than one in every 7 million unique queries where an overview appeared.
Of the testing before launch, she wrote: “There’s nothing quite like having millions of people using the feature with many novel searches.”
Not everyone accepted the framing. The Guardian noted that while many viral answers came from bizarre searches, others appeared to come from more reasonable ones, and that several AI experts said the problems pointed to wider issues with AI’s ability to gauge factual accuracy.
Google said it had made more than a dozen technical changes. It built better detection for nonsensical queries, limited satire and humor, and cut back on user-generated content in answers that could mislead. It also added restrictions on when an overview would appear. Fewer searches would now come back with an AI-written summary at all.