開發者如何用一條 robots.txt 規則退出 AI 訓練(2027)
One robots.txt Rule to Opt Out of AI Training: What Developers Need to Know (2027)
Google 於 2026 年 10 月 30 日宣佈,計劃根據 IETF AIPREF 工作組的工作,在 robots.txt 支援共用的 AI 訓練控制規則。
On October 30, 2026, Google announced plans to support a shared AI training control in robots.txt, based on work from the IETF's AIPREF working group. Coverage the same day reported that Microsoft and OpenAI will honour it too, and that Anthropic has started building support.
If you build or maintain websites, here's what it means and what to do (all free).
The rule
User-Agent: __usage_ai-training
Disallow: /
Instead of a crawler name, the User-Agent line holds a policy token. Normal robots.txt path matching applies, so you can opt out site-wide and still allow a folder:
User-Agent: __usage_ai-training
Disallow: /
Allow: /press-kit/
Timeline (it's not live yet)
- Microsoft: from February 1, 2027
- Google: "early 2027" per its crawler documentation note; the blog says 2027
- OpenAI: "in the coming months"
- Anthropic: implementation started, no date
It's an IETF draft, so the exact token or details could still change.
The Google-Extended catch
Google says that if you use the shared token to opt out and don't explicitly specify Google-Extended, it will treat that as if you disallowed Google-Extended. Google also says sites that allow Google-Extended are eligible to appear and receive clicks via grounding in Gemini Apps.
So if you want to opt out broadly but still be cited in Gemini, you'd add an explicit allow, knowing that it also lets Google use the content for Gemini:
# Opt out of AI training with all participating AI providers
User-Agent: __usage_ai-training
Disallow: /
# Optional: still allow Google's Gemini (training + citations in Gemini Apps)
User-Agent: Google-Extended
Allow: /
Checklist
- Open
yoursite.com/robots.txtand see what's already there. - Decide: protect your content from training, or stay visible in AI answers?
- If opting out, add the shared token group.
- Decide on Google-Extended explicitly, so nothing happens by default.
- Google Search rankings aren't affected. Re-check the spec in early 2027.
What would you choose for your projects: block, allow, or a mix by folder?
來源:DEV Community · dev.to