OpenAI, Google, Microsoft, Anthropic Back AI Opt-Out Signal

Four of the world's biggest AI companies just agreed on something they'd spent two years fighting over: a single way for websites to say "don't train on my content." On October 30, OpenAI, Google, Microsoft and Anthropic said they'll start honoring one shared signal in a website's robots.txt file — the small text file that tells web crawlers what they can and can't touch — instead of making publishers juggle a different opt-out for every company's bot.

What's actually changing

Right now, if a news site wants to stop its articles from being used to train AI models, it has to block each crawler separately — OpenAI's GPTBot, Google's Google-Extended, Anthropic's ClaudeBot, and so on — using rules that aren't even interpreted the same way by every company. The new approach, built on an IETF (the body that sets core internet standards) draft called AI Preferences, lets a publisher set one instruction that all participating companies agree to read the same way: train on this, or don't.

Anthropic says it already supports the new signal. OpenAI plans to roll it out "in the coming months." Google is targeting early 2027. Microsoft's Bing is the most specific, with two dates: it starts honoring the new preference on February 1, 2027, and gives publishers until March 1, 2027 to migrate off the old, patchwork rules. Microsoft is also splitting the opt-out in two — training versus "grounding" (letting an AI chatbot cite or summarize your content in real time) — and says sites will keep showing up in ordinary Bing search results no matter which box they tick.

• One signal, four companies — instead of separate rules per crawler, publishers set one preference.
• Staggered rollout — Anthropic now, OpenAI "coming months," Google and Microsoft in 2027.
• Still voluntary — robots.txt has never been legally enforceable; it only works if companies choose to respect it.

A single checkbox doesn't fix a system that let the ingestion happen years before anyone built the checkbox.

Why Indian publishers have heard this before

This is, almost word for word, the argument at the center of a Delhi High Court case that's been running since 2025. The Digital News Publishers Association, whose roughly 20 members include NDTV, Network18, India Today Group, Zee News and Dainik Bhaskar, has accused OpenAI of what its filing calls "wilful scraping" — pulling and reproducing copyrighted Indian journalism to train ChatGPT, including content from sites that had already set robots.txt rules asking it not to. The DNPA's own secretary general has said that 70 to 80 percent of its member sites have now restricted AI crawlers, and that blocking the door after training data has already been copied doesn't undo the copying.

That's the gap a shared robots.txt signal doesn't close. It standardizes the mechanism going forward; it does nothing about what's already inside a model's training data, and it carries no penalty if a company's crawler ignores it anyway — which is exactly what Indian publishers allege happened the first time. India's copyright law, written in 1957, has no AI-specific provision at all, so publishers have been left arguing existing infringement law in court rather than pointing to a rule built for this problem. It's a different legal question from the one answered by India's own DPDP Act, which governs personal data, not copyrighted journalism — but the comparison is telling: India regulates what happens to your phone number more precisely than what happens to a newsroom's published reporting.

What to actually watch

For Indian developers and startups building on top of these models, the practical question is whether "AI Preferences" ends up as a real technical standard they can implement once, or another voluntary gesture that different companies interpret differently. For publishers, the more useful signal will be what courts — not crawlers — eventually decide about content already used without permission.

1. Whether Google and Microsoft actually ship their 2027 timelines on schedule.
2. Whether the Delhi High Court case forces a disclosure of what training data OpenAI already holds.
3. Whether India's copyright law gets an AI-specific amendment, or keeps getting stretched to cover a problem it wasn't written for.

A shared signal is a real, useful piece of infrastructure — it's just not the same thing as an answer to who gets paid for the journalism that trained these models in the first place. For Indian publishers watching this fight from a courtroom instead of a settlement table, that distinction is the whole story.