OpenAI, Google, Microsoft, Anthropic draft AI training preference spec

On November 1, 2026, IT Home reported that OpenAI, Google, Microsoft and Anthropic had announced plans to adopt a new standard developed by Anthropic engineering lead Kevin Kelley and submitted to the IETF. Known as the machine-readable training preference signal draft, it is implemented through the robots.txt file on a website. Content publishers can use this mechanism to declare their preferences on AI training use of their content to multiple AI companies at once, rather than configuring separate rules for each company as they have had to do until now. The robots.txt file has long been used to tell web crawlers which parts of a site may be crawled and which may not, covering search indexing, web archiving and AI model training. In its coverage, Anthropic had already adopted the standard, OpenAI planned to implement it within the following months, and Google and Microsoft were expected to support it in 2027.

The next day, November 2, 2026, Landian provided more detail on each company's timeline. Anthropic appeared to have already started supporting the draft; OpenAI said it would implement it within the coming months; Google planned to begin supporting it in early 2027, though the exact timing would depend on further preparation work; and Microsoft stated that it would comply starting February 1, 2027. Under the draft, a site administrator adds a set of rules to robots.txt to declare whether site content may be used for AI training, with settings that can be defined per directory. It is worth noting that these declarations express the site's preference regarding content use only. Adding them alone does not directly block crawlers from accessing pages; to restrict a specific crawler, a site still needs to configure traditional robots.txt rules. Separately, training use and citation of web content in AI answers are different use cases, and Microsoft offers an additional AI Grounding control for its own services, letting sites decide whether content may be referenced when AI generates answers; that control operates at the page level. IT Home's report also mentioned that Meta spokesperson Andy Stone had commented, though the full content of that remark is not available in the available material. The proposal remains an Internet draft, with each company due to roll it out according to the timelines above.