谷歌、微软、OpenAI支持统一robots.txt信号：网站一处设置可拒绝超级智能（SI）训练
谷歌10月30日在其 Search Central 官方博客宣布，计划支持一项统一的 robots.txt 超级智能（SI）训练信号：网站今后只需在 robots.txt 里写一处共享「政策令牌」，就能同时向多家参与的超级智能（SI）公司表达是否允许用本站内容训练模型，不必再逐家公司分别设置。据谷歌该篇由 John Mueller 撰写的博文，这项机制源自互联网工程任务组（IETF）旗下 AIPREF 工作组的工作。
这项机制写在同日提交的一份互联网草案《机器可读训练偏好信号》里，草案由 Anthropic 的 Kevin Kelley 执笔，核心是名为 __usage_ai-training 的政策令牌。谷歌表示：若网站用该令牌拒绝训练、又未单独提及 Google-Extended，谷歌计划按「拒绝 Google-Extended」来解读；Google-Extended 是谷歌自2023年起提供的控制项，覆盖未来 Gemini 模型的训练以及在 Gemini 应用与 Vertex AI 中的落地引用。谷歌目标在2027年支持这项新信号。
另外两家同日跟进。OpenAI 宣布将尊重这一共享信号，计划在未来数月内实施，现有的各家专用控制项继续有效。微软必应宣布自2027年2月1日起认读这项共享训练偏好，并新增按页面设置的落地引用偏好；自2027年3月1日起，NoArchive 与 NoCache 标记不再被读作训练或落地引用信号，仍在用旧标记的网站需改用新设置。
需要留意的是，这项信号只管「训练」一件事，不影响搜索引擎的抓取与收录；且它目前仍是个人名义提交的互联网草案，尚不是 IETF 的正式标准。网站现有的各家专用控制项在过渡期内照常有效。
Google, Microsoft and OpenAI Back a Shared robots.txt Signal to Opt Out of Super Intelligence (SI) Training
Google announced on October 30, in a post on its official Search Central blog, that it plans to support a shared super intelligence (SI) training control in robots.txt: websites will be able to use a single shared policy token to tell multiple participating SI providers at once whether their content may be used for model training, instead of setting separate rules for each company. According to the post, written by John Mueller, the mechanism builds on work by the AIPREF working group at the Internet Engineering Task Force (IETF).
The mechanism is set out in an Internet-Draft titled Machine-Readable Training Preference Signal, submitted the same day and authored by Kevin Kelley of Anthropic; its core is a policy token named __usage_ai-training. Google says that where a site opts out of training through the shared token and does not explicitly specify Google-Extended, Google plans to interpret that as disallowing Google-Extended, the control it has offered since 2023, which covers training of future Gemini models and grounding in Gemini Apps and on Vertex AI. Google aims to support the new control in 2027.
OpenAI made a parallel commitment the same day, saying it will honor the shared signal, aims to implement it in the coming months, and will keep its existing crawler controls in place. Microsoft's Bing said it will honor the shared training preference from February 1, 2027, and is adding a separate page-level grounding preference; from March 1, 2027, it will stop reading NoArchive and NoCache as training or grounding signals, so sites still relying on those tags will need to switch to the new settings.
The signal covers training only and does not affect crawling or inclusion in search. It also remains an individually submitted Internet-Draft rather than a formal IETF standard for now, and existing company-specific controls continue to work during the transition.
来源：GSM·美国（原文链接）
图片来源：图：谷歌总部 Googleplex（加利福尼亚州 Mountain View）。Grendelkhan 摄，CC BY-SA 3.0，经 Wikimedia Commons。 / The Googleplex, Google's headquarters in Mountain View, California. Photo by Grendelkhan, CC BY-SA 3.0, via Wikimedia Commons.
GSM Global · 本篇英文版 English edition of this story
