語言模型任務組閤中的脆弱性與隱式預算分配
On the Fragility of Tasks under Composition: Implicit Budgeting in Language Models
一項涵蓋開放權重模型及 Claude Opus 5 的研究發現,語言模型在單一提示中組合多項任務時,即使各項任務單獨均能可靠完成,錯誤仍會顯著增加。這種退化不能完全以獨立錯誤累積解釋;組合任務時,模型分配給每項任務的 token 明顯少於單獨處理時,所需解題過程較長的任務因此更脆弱。按脆弱性將任務分組,可在多項數學及語言任務中減少 43% token 用量,同時維持準確率。
Published on Oct 19
Authors:
,
,
,
Abstract
Modern language models are increasingly asked to handle several tasks within a single prompt. Doing so requires solving each task correctly while integrating the resulting answers into a coherent response. In this paper, we identify a fundamental limitation in this process, where models make substantially more errors when tasks are composed, even when they can solve every constituent task reliably in isolation. This degradation cannot be fully explained by the accumulation of independent errors. Across both open-weight models and the frontier model Claude Opus 5, our findings instead point to an implicit thinking budget underlying this effect. In particular, under composition, models use substantially fewer tokens per task than in isolation, even when the full sequence remains well within the context window. Tasks that require longer solution traces in isolation are consequently more fragile under composition, i.e., models make more errors on prompts containing these tasks. In isolation, non-fragile tasks receive more reasoning than they need and waste compute. However, by grouping tasks according to their fragility, the implicit budgeting can be manipulated to reduce token usage by 43\% across a range of math and linguistic tasks while also preserving accuracy.
View arXiv page View PDF Project page GitHub Add to collection
Get this paper in your agent:
hf papers read 2610.27190
Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash
Models citing this paper 0
No model linking this paper
Cite arxiv.org/abs/2610.27190 in a model README.md to link it from this page.
Datasets citing this paper 0
No dataset linking this paper
Cite arxiv.org/abs/2610.27190 in a dataset README.md to link it from this page.
Spaces citing this paper 0
No Space linking this paper
Cite arxiv.org/abs/2610.27190 in a Space README.md to link it from this page.
Collections including this paper 0
No Collection including this paper
Add this paper to a collection to link it from this page.
來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co