Why AI text watermarks may be easy to defeat

Text watermarks intended to identify AI generated content can be removed with simple editing or paraphrasing, according to an analysis of upcoming European rules. Sean Goedecke writes in his personal blog that this creates a basic challenge for providers preparing for the EU AI Act’s transparency requirements.

The Act’s Article 50 is widely expected to require providers to make AI output detectable. That could lead companies to add hidden markers to generated text and offer tools that check whether a passage carries their marker. Anthropic has just announced this for Claude. Other vendors will probably follow.

Two approaches, similar limits

Google has publicly described a text watermarking system called SynthID. It subtly influences which likely word or word fragment an AI model selects while producing a response. A detector can then assess whether the resulting text shows a statistically unusual pattern. The method aims to preserve readable output while giving providers a relatively inexpensive way to check content.

Goedecke also points to a simpler possible technique: replacing ordinary characters, especially spaces or punctuation, with visually similar Unicode characters. Such changes can encode a pattern without changing what readers see. He notes that unusual Unicode characters have appeared in copied output from major AI tools, although this alone does not prove they are used as watermarks.

Both methods have a central weakness. Unicode markers can be removed by normalising characters. Statistical watermarks such as SynthID can be weakened or erased when a person or another AI system rewrites the text. Even a lightweight local model may be sufficient to paraphrase content, he argues.

That leaves a difficult policy question. The EU’s related guidance calls for markings that are hard to separate from content, while also seeking technical interoperability. Publishing or standardising watermark methods could make them easier to evade.

Signed metadata systems such as C2PA may help with files such as images, video, PDFs or packaged documents. But they do not readily cover ordinary text copied from a chat window into an email, document or website.