跳到正文
HuggingFace Daily Papers(社區熱門論文)·· 2026-10-22AI 評分45

繼承而非創造:追溯推理模型中 {wait} 標記的來源

Inherited, Not Invented: Tracing the Provenance of Wait in Reasoning Models

AI 導讀

研究追溯推理模型中 {wait} 標記的來源,發現強化學習提升推理準確度並不需要這個標記;標記主要源自訓練資料,而非強化學習。基礎模型原本沒有 {wait} 時,零強化學習幾乎不會產生它,但加入強化學習後 MATH-500 準確度仍大幅提升。研究亦指出,單看標記頻率不足以判斷推理變化,自動評審的修訂指標可能與人工標註趨勢相反。

正文

Published on Oct 22

Authors:

,

,

,

,

,

,

,

Abstract

Reinforcement learning improves reasoning accuracy without requiring a specific wait marker, which originates from training data rather than RL, and automated evaluation metrics can misrepresent reasoning trends.

Reinforcement learning (RL) is credited with teaching reasoning models to write {wait} before reconsidering a step, yet large RL accuracy gains do not require the word. This view can confuse what RL discovers with what a model inherits from its training data. We trace the word through open models' training histories, reading it with label-free indices of frequency, position, form and usage and an injected-error test, all computed from sampled outputs. We find that zero-RL almost never coins the marker: on bases that lack the word, RL raises MATH-500 accuracy substantially while {wait} stays near absent. It arrives with training data: mid-training in OLMo-3, and fine-tuning on reasoning traces installs it in most models whose base lacks it. Where the base already uses it, both released RL-Zero recipes (OLMo-3, MiMo) multiply it, whereas KL-free GRPO from the same base does not; OLMo-3's recipe wears down its form and makes it a more sensitive but more liberal alarm; RL after supervised fine-tuning barely changes it, and our GRPO recipe erases a marker seeded by fine-tuning. Finally, we show that marker frequency alone does not indicate RL-induced reasoning and caution against LLM-as-a-judge revision metrics, as automated judges can reverse human-labelled trends.

View arXiv page View PDF GitHub Add to collection

Get this paper in your agent:

hf papers read 2610.30882

Don't have the latest CLI?

curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 0

No model linking this paper

Cite arxiv.org/abs/2610.30882 in a model README.md to link it from this page.

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2610.30882 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2610.30882 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.

來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co