跳到正文
HuggingFace Daily Papers(社區熱門論文)·· 12 天前AI 評分47

Re-Search:透過雙人提問遊戲剖析 LLM 的研究構思

Re-Search: Dissecting LLM Ideation Through a Two-Player Questioning Game

AI 導讀

Re-Search 以「二十問」形式評估 LLM 的研究構思,發現模型的進展幾乎都集中於首 30 個問題。研究以 200 篇 ICML 2026 論文測試六款近期 LLM,每局共 100 個問題;多數模型其後進展停滯。結果顯示,瓶頸在於生成而非判斷,為自動化研究系統加入外部構思和問題或比單純提升模型生成能力更有效。

正文

Published on Nov 10

Authors:

,

,

,

,

Abstract

A Twenty Questions-style benchmark for research ideation shows that large language models make most progress early, can recognize good ideas or questions but often cannot generate them, and may benefit from externally sourced candidates in automated research systems.

Research ideation is iterative: researchers refine their ideas step by step based on feedback. Yet existing evaluations obscure this search process, making it difficult to understand how LLMs explore the space of research ideas and what limits their progress. We introduce Re-Search, which adapts the format of Twenty Questions to research ideation: given only the research problem of a published paper, a model must rediscover the paper's core idea by asking yes-or-no questions. Built from 200 ICML 2026 papers, Re-Search tracks how a model's idea evolves as it asks questions and receives binary oracle feedback, isolating ideation from execution. Across six recent LLMs, nearly all progress happens within the first 30 of 100 questions, which largely decide how the game ends; after that, most models plateau. Through controlled interventions on the candidate questions and the hypothesis space, we find that the bottleneck is generation, not judgment: models recognize a good idea or a good question when one is offered, but often cannot produce it on their own. We further analyze the ideation traces and find that models fail by either over-exploiting one direction or over-exploring too many. These findings point to concrete design principles for autoresearch systems: since models can judge candidates they cannot produce, enriching the search with externally sourced ideas and questions may be more effective than improving any single model's generation ability.

View arXiv page View PDF Add to collection

Get this paper in your agent:

hf papers read 2611.14487

Don't have the latest CLI?

curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 0

No model linking this paper

Cite arxiv.org/abs/2611.14487 in a model README.md to link it from this page.

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2611.14487 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2611.14487 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.

來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co