A fake hot-dog champion exposed a major flaw in AI answers

AI chatbots from some of the world’s biggest tech companies can be manipulated into spreading misinformation through a simple technique. Thomas Germain reports for the BBC that a single, well-crafted blog post published almost anywhere online can be enough to influence what tools like ChatGPT, Google Gemini and Google’s AI Overviews tell the public.

Germain demonstrated the problem himself by publishing an article on his personal website claiming to be a world-champion competitive hot-dog eater. Within a day, major AI tools were repeating the false claim as fact.

The same technique has been used for more serious purposes, including dismissing health concerns about medical supplements and influencing financial advice provided by Google’s AI.

Google recently updated its spam policies to confirm that attempts to manipulate AI responses violate its rules. Offending websites risk being removed from or downranked in search results. The company insists the update is a “clarification” of existing efforts rather than a new approach.

Search experts Lily Ray and Harpreet Chatha have observed additional quiet changes. These include AI tools removing self-promotional sources from answers and adding more caveats to responses. However, Chatha warns that manipulators adapt quickly. “Google is playing whack-a-mole,” he says. “They’re announcing [the policy update] to deter people, but the tactics will just move.”

Ray advises the public to treat AI answers with scepticism until better safeguards exist. “You should assume that you’re being manipulated until they have better systems in place,” she says.