A coalition of major web publishers, including Reddit, Yahoo, and Quora, has launched a new system called Really Simple Licensing (RSL). The standard aims to make AI companies pay for using online content to train their artificial intelligence models.
The RSL Standard builds on the existing `robots.txt` protocol, a file that gives instructions to web crawlers. The Verge reports that publishers can now use this file to embed specific licensing terms. These can include subscription models, a “pay-per-crawl” fee for each time a bot visits a site, or a “pay-per-inference” fee, which compensates the publisher when an AI uses their content to form an answer.
The initiative is led by the RSL Collective, a new rights organization. Co-founder Eckart Walther stated the goal is to create “a new, scalable business model for the web”. The collective aims to simplify the licensing process, allowing any creator to get paid without having to negotiate individual deals with AI companies.
The system’s success depends on adoption by these AI companies. RSL itself does not block bots, but the collective is partnering with content delivery networks like Fastly to enforce access based on licensing agreements. Co-founder Doug Leeds said the group will also collectively enforce licenses and distribute fees, comparing the model to music rights organizations like ASCAP.
A similar initiative is Cloudflare’s “Pay Per Crawl”. Read also more about why Cloudflare’s CEO wants AI crawlers to pay websites.