跳到正文
HuggingFace Daily Papers(社區熱門論文)·· 10 天前AI 評分45

思考的價值:混合決策模型如何學習何時進行推理

The Value of Thinking: Learning When to Reason in Hybrid Decision Models

AI 導讀

混合決策模型(HDM)透過自我路由區塊,在單次前向傳遞的隱式思考與自回歸推理之間作出選擇,並以 CDPO 訓練路由區塊。在 JevBench 上,HDM-4B 得分為 0.857,對比 Jev 的 0.851(+0.7%);中位延遲為 0.179\,s,對比 0.652(-72.5%)。在 MiniWoB 上,HDM-27B 將 token 成本降低 80.1%。

正文

Published on Nov 12

Authors:

,

,

,

,

Abstract

A hybrid decision model routes between implicit single-forward-pass answering and full autoregressive reasoning via a self-routing block trained with calibrated decision policy optimization, reducing token and latency costs while preserving accuracy.

Existing adaptive-reasoning methods shorten the explicit reasoning a model writes, or skip it by switching modes; either way the answer is still written out token by token. Rather than deciding how much to think, we explore a different paradigm, zero-or-T thinking: a single model either answers from the implicit thinking inside one forward pass, generating no token, or performs full autoregressive reasoning at a cost of T tokens when that is worth paying. We realise it in a hybrid decision model (HDM), where one frozen backbone serves both modes and a lightweight self-routing block chooses between them. The block estimates the value of thinking, the utility the slow mode is expected to add over the fast one, from what that implicit thinking has already produced, and the slow mode runs only where that value exceeds a price λ set per request. Where no per-decision label exists, as in episodic agentic tasks, we introduce Calibrated Decision Policy Optimisation (CDPO), a reinforcement-learning recipe that trains the block from episode-level returns. We evaluate HDM on MMLU-Pro, AGIEval, GPQA-Diamond, JevBench, BANKING77 and MiniWoB. Specifically, on JevBench HDM-4B scores 0.857 against Jev's 0.851 (+0.7%) at a median 0.179\,s against 0.652 (-72.5%). On the agentic venue MiniWoB HDM-27B cuts token cost by 80.1%.

View arXiv page View PDF GitHub Add to collection

Get this paper in your agent:

hf papers read 2611.16961

Don't have the latest CLI?

curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 1

fredreick/HDM-blocks

Updated about 5 hours ago

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2611.16961 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2611.16961 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.

來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co