Why Google (and other companies) could pay dearly for their AI’s mistakes

A German court has ruled that Google is legally responsible for the content of its AI-generated search summaries. Bruce Schneier writes for Schneier on Security that the court rejected Google’s argument that users should verify AI output themselves, ruling instead that the summaries are “an expression of Google’s business activities.”

The decision reignites a long-running debate over how to classify internet platforms: as neutral carriers, like phone companies, or as publishers responsible for their content, like newspapers. Section 230 of the US Communications Decency Act has long let platforms claim both roles depending on convenience. But AI systems, Schneier argues, blur this line further because they rewrite and synthesize third-party content rather than simply displaying it, acting more like an editor than a mere conduit.

Millions of errors, real consequences

The stakes are significant. Tests cited in the report found Google’s AI Overview feature makes errors roughly 10 percent of the time, translating to an estimated 16,000 faulty summaries per second globally. One case already in court: Canadian musician Ashley MacIsaac is suing after Google’s AI falsely labeled him a sex offender.

Schneier draws a parallel to a 2024 case in which a Canadian court held Air Canada liable for its chatbot’s promises, rejecting the airline’s claim that the bot was a separate legal entity.

For companies deploying AI agents, from customer service bots to Visa’s planned AI purchasing assistants, the message seems clear: liability cannot be outsourced to algorithms. If businesses want to use AI to write, advise or negotiate on their behalf, they must own the consequences, just as they would for any human employee.