跳到正文
HuggingFace Daily Papers(社區熱門論文)·· 2026-10-20AI 評分40

DeCert:生成式獎勵模型的決策證書監督與理由評估

Judging the Justification: Decision Certificate Supervision for Generative Reward Models

AI 導讀

DeCert 評估並改善生成式獎勵模型的判斷理由,透過檢查理由有效性及判決一致性,在毋須參考解釋的情況下產生偏好監督。跨兩個模型系列及多個分佈內外基準的實驗顯示,DeCert 優於強勁的生成式評判基線,並將位置一致準確率最多提高 14.8 個百分點。

正文

Published on Oct 20

·

Submitted by

sgq

on Oct 26

Authors:

,

,

Abstract

DeCert evaluates and improves generative reward model rationales by assessing justification validity and verdict consistency without reference explanations, yielding stronger preference supervision.

Generative reward models (GenRMs) generate natural language rationales to justify preference judgments, yet a correct verdict does not guarantee a sound justification. Existing approaches often rely on reference explanations to supervise rationales, but obtaining such references can be costly and does not necessarily establish whether generated rationales support the resulting verdict. To address these limitations, we propose DeCert, a framework that evaluates the validity and verdict consistency of generated rationales without requiring reference explanations. Crucially, DeCert translates these assessments into preference supervision that accounts for both verdict correctness and justification quality. In particular, when multiple sampled judgments share the same correct verdict, differences in certificate quality enable DeCert to construct rationale preference pairs that distinguish stronger from weaker justifications. Extensive empirical studies across two model families and multiple in- and out-of-distribution benchmarks demonstrate that DeCert consistently outperforms strong generative judge baselines while improving rationale quality. Moreover, DeCert effectively mitigates positional bias, improving position-consistent accuracy by up to 14.8 points. These results highlight the importance of supervising not only which verdict is correct, but also how that verdict is justified.

View arXiv page View PDF GitHub 2 Add to collection

Get this paper in your agent:

hf papers read 2610.27528

Don't have the latest CLI?

curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 1

gqbond/DeCert-Llama-3.1-8B

Text Generation •

8B •

Updated 6 days ago

•

149

•

1

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2610.27528 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2610.27528 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.

來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co