Summary
OpenAI says it will honor a shared robots.txt signal for website operators to express preferences about using their content for foundational generative AI training. The company plans to implement it in the coming months, alongside its existing crawler controls.
OpenAI announced on October 30, 2026, that it will honor a shared signal in robots.txt through which website operators can express preferences about using their content for foundational generative AI training. The company says it plans to implement the signal in the coming months and will share details when it does.
How the shared signal changes website controls
Website operators have typically had to identify individual crawlers and maintain separate rules for each provider. OpenAI says a common signal could let them communicate the same preference to multiple participating AI providers, reducing that administrative work.
Robots.txt is a site-level file used to communicate instructions to web crawlers. OpenAI describes this new signal as a voluntary way to express preferences about AI training. It is encouraging other AI providers to honour it, so the shared approach depends on providers choosing to respect the signal.
The change will sit alongside, rather than replace, OpenAI’s existing robots.txt crawler controls. The company says those controls will continue to apply and be respected.
Connected to ongoing standards work
OpenAI says the effort draws on the Internet Engineering Task Force’s ongoing AI Preferences working group. The company is joining other participants in announcing the shared signal and says it will continue to take part in IETF standards discussions as the work evolves.
For website operators, the practical change will depend on OpenAI’s implementation and on other providers adopting the signal. OpenAI has said implementation is planned for the coming months, with further details to follow.