跳到正文
HuggingFace Daily Papers(社區熱門論文)·· 2026-10-09AI 評分35

GraphDecisionBench:System One 模型的圖任務基準測試

GraphDecisionBench: Benchmarking System One Models on Graph Tasks

AI 導讀

這項研究提出 GraphDecide,一項與模型無關的基準測試,用於診斷模型在圖決策任務上的表現。研究評估 Jev 及相關選項式模型,並與大語言模型基線比較,涵蓋 14 種模型介面配置。Jev 的結果顯示,準確識別鄰接關係不代表具備更廣泛的結構正確性,圖文共同輸入亦未能穩定改善預測,而可行的構造不等於高質素解答。

正文

Published on Oct 9

Authors:

,

Abstract

Large language models (LLMs) are increasingly explored for graph understanding and decision-making, while System One models such as Jev select directly from supplied options. However, the capabilities of System One models on graph-related tasks remain unclear. We introduce GraphDecide, a model-independent benchmark that combines structural task profiles, matched graph-text input contrasts and heuristic-proposal controls to diagnose graph decision performance. We evaluate Jev and related choice-based models alongside language-model baselines, covering fourteen model-interface configurations. Jev's results illustrate the benchmark's central distinctions: accurate adjacency recognition does not guarantee broader structural correctness, joint graph-text input does not consistently improve prediction, and feasible construction does not establish high solution quality. Its task contracts, candidate interfaces and scoring rules support comparison across native selectors and language-model adapters. Code and aggregate results are available at https://github.com/VictorYXL/JevGraphBench.

View arXiv page View PDF GitHub Add to collection

Models citing this paper 0

No model linking this paper

Cite arxiv.org/abs/2610.06354 in a model README.md to link it from this page.

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2610.06354 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2610.06354 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.

來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co