跳到正文
HuggingFace Daily Papers(社區熱門論文)·· 2026-10-03AI 評分41

Questioning the Questions: Sustaining Self-Evolution in Reasoning Models

Questioning the Questions: Sustaining Self-Evolution in Reasoning Models

AI 導讀

R-Quest 利用問題有效性及新穎度回饋引導推理模型自我演化,應對無效問題增加及數學問題重複造成的表現崩塌。在兩個模型家族的數學推理、通用領域推理及程式碼生成共 12 項基準測試中,R-Quest 取得最高平均表現。連續十輪自我演化期間,R-Quest 維持穩定的表現提升,並在最後一輪達到峯值,較 R-Zero 高 17.32 分。

正文

Abstract

Self-evolving reasoning models learn from their own generated questions, yet repeated self-training can lead to performance collapse. In this paper, we investigate why performance deteriorates over successive rounds and how to sustain self-evolution. Our analysis identifies two recurring quality problems in self-generated questions: invalid questions and repeated variants of the same mathematical questions. First, invalid questions become more prevalent across rounds, and answer-consistency filtering further increases their proportion in training data. Second, existing question diversity controls based on lexical similarity can miss mathematically equivalent questions expressed in different ways, which leads to question diversity collapse in later training rounds. Building on these findings, we introduce R-Quest, which uses question validity and novelty feedback to guide self-evolution. We first train the solver to recognize and reject invalid questions, then use its judgments to guide questioner rewards and filter solver training data. To avoid question repetition, we use a frozen base model to compare sampled question pairs and provide novelty feedback. Empirically, our method consistently achieves the highest average performance on 12 benchmarks in mathematical reasoning, general-domain reasoning, and code generation across two model families. Additionally, R-Quest maintains stable performance gains over ten rounds of self-evolution, peaking in the final round and outperforming R-Zero by 17.32 points.

View arXiv page View PDF Add to collection

Models citing this paper 1

jinyuan222/R-Quest-Qwen3-4B-Validity-RL

Text Generation •

4B •

Updated 2 days ago

•

415

Datasets citing this paper 1

jinyuan222/rzero-validity-rl-terra-v1-clean-v1

Viewer •

Updated 2 days ago

•

2.23k

•

89

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2610.04299 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.

來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co