OpenAI’s Atlas and Perplexity’s Comet pose new challenges for publishers trying to control AI access to their content. These AI browsers can retrieve paywalled articles that traditional AI interfaces cannot access, the Columbia Journalism Review reports.
The browsers succeed because they appear identical to human users in website logs. While automated crawlers identify themselves with digital IDs that publishers can block, AI browsers like Atlas and Comet disguise themselves as regular Chrome sessions. This makes blocking them difficult without also preventing legitimate human access.
Many publishers use client-side paywalls that load full text but hide it behind subscription prompts. AI agents can read this hidden content. Even server-side paywalls become vulnerable once a logged-in user allows the AI browser to access their account.
OpenAI states it does not train models on Atlas browsing data unless users opt into browser memories. However, uncertainty remains about how much the company learns from paywalled content that users unlock for agents.
The Columbia Journalism Review found that Atlas appears to avoid content from publishers currently suing OpenAI. When prompted to summarize blocked articles, Atlas uses workarounds. It reconstructs content by gathering information from tweets, syndicated versions and related coverage. For New York Times articles, it instead summarizes reporting from alternative outlets, several of which have licensing deals with OpenAI.
Publishers face a dilemma. Even when they successfully block AI agents from their content, the agents simply redirect users to competing sources. Traditional defenses like paywalls and crawler blockers no longer provide adequate protection against AI systems accessing news content.
