跳到正文
HuggingFace Daily Papers(社區熱門論文)·· 2026-10-22AI 評分52

聯合評分還是逐選項評分?17 million 至 1 billion 參數的封閉集合決策緊湊編碼器

Joint or Per-Option? Compact Encoders for Closed-Set Decisions from 17 Million to 1 Billion Parameters

AI 導讀

研究比較封閉集合決策中的聯合評分與逐選項評分,結果顯示在 32 million 參數設定下,逐選項方式表現較佳。在 37 個基準決策套件的留出樣本上,三個 seed 試驗中逐選項方式按相同 optimizer steps 和近似相同訓練計算量,分別高出 5.8 和 2.8 個經機會校正的技能分,但需要多 5 至 13 倍編碼器運算次數。

正文

Published on Oct 22

Authors:

Abstract

Many language-model pipelines end in a closed-set decision, such as selecting a tool. Prompting a large language model for it is slow and order-sensitive. We ask how small an encoder for such decisions can be and whether it should read the options jointly or separately. In a three-seed study at 32 million parameters, per-option scoring beats joint scoring on a held-out sample of a 37-benchmark decision suite by 5.8 points of chance-corrected skill at matched optimizer steps and by 2.8 at approximately matched training compute; two external benchmarks agree, though not significantly on their unseen tasks. Crossing reader with initialisation shows that the advantage requires a reranker initialisation, whose query-candidate pair input matches per-option reading; from the raw encoder the two readers are indistinguishable. Per-option scoring needs 5 to 13 times more encoder passes: at most twice the median latency, but more than four times the mean. Of six released joint encoders (17 million to 1 billion parameters), the 68- and 150-million-parameter models exceed a 435-million-parameter DeBERTa-v3 zero-shot classifier by 4.9 and 5.9 accuracy points on 209 overlap-audited System1-Arena tasks and are at least on par with it on the 145 tasks unseen in training. The smallest answers a median request in 14 milliseconds on 16 processor threads, against 728 for the classifier. Reordering options changes 12 to 23% of the joint models' answers (per-option scoring: none). Up to 68 million parameters, they lead their size range on a public leaderboard's private, mainly in-distribution tests.

View arXiv page View PDF Project page GitHub 0 Add to collection

Get this paper in your agent:

hf papers read 2610.30970

Don't have the latest CLI?

curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 8

tardellirs/crivo-150m

Text Classification •

0.1B •

Updated 4 days ago

•

477

•

14

tardellirs/crivo-68m

Text Classification •

68.4M •

Updated 4 days ago

•

91

•

3

tardellirs/prumo

527 •

Updated 4 days ago

•

2

tardellirs/crivo-32m

Text Classification •

32M •

Updated 4 days ago

•

175

•

1

Browse 8 models citing this paper

Datasets citing this paper 1

tardellirs/decision-index-results

Updated 4 days ago

•

290

Spaces citing this paper 3

Collections including this paper 1

來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co