The biggest names in AI are backing a new way to opt out of scraping
A unified opt-out standard across major AI labs simplifies how web publishers control the use of their content for AI training and could affect the availability of public web data for future models.
Covered by 2 sources · first reported by OpenAI
What happened
OpenAI, Google, Microsoft, and Anthropic have agreed to adopt a unified standard allowing publishers to opt out of AI web scraping across multiple companies simultaneously. Proposed through the Internet Engineering Task Force by an Anthropic technical lead, the shared policy token will operate within a website's existing robots.txt file.
Who it’s about
- OpenAI
- Microsoft
- Anthropic
- Kevin Kelley
Coverage, oldest first
More on search
Get a brief picked for you
Sign in to follow topics and companies, save stories and search the archive.
Sign in