跳到正文
HuggingFace Daily Papers(社區熱門論文)·· 29 天前AI 評分40

Mycelium:跨來源、類型與粒度的幻覺檢測元研究

Mycelium: A Meta-Study of Hallucination Detection Across Provenance, Typology, and Granularities

AI 導讀

Mycelium 將幻覺檢測視為受基準測試條件影響的測量問題,並按來源、類型及粒度分析現有基準測試。研究指出,標註邊界差異會影響跨度層級的評估結果,並提出以涵蓋度為基礎的 Span Coverage F1,作為補充指標。

正文

Published on Oct 24

Authors:

Abstract

Hallucination detection is analyzed as a benchmark-conditioned measurement problem across provenance, typology, and granularity axes, with a proposed Span Coverage F1 metric to address span-level evaluation variability.

Hallucination detection is increasingly important for evaluating and deploying large language models. However, existing methods are often compared across benchmarks that define and measure hallucination in substantially different ways, making reported scores difficult to interpret across settings. We study hallucination detection as a benchmark-conditioned measurement problem and organize existing benchmarks along three axes: provenance, which describes how model outputs and evidence conditions are obtained; typology, which captures the factuality and faithfulness failures represented by benchmark labels; and granularity, which specifies the unit at which hallucinations are annotated, predicted, and evaluated. Using this framework, we analyze how benchmark construction, annotation, and evaluation choices shape the underlying prediction task, and relate detector mechanisms and assumptions to the settings in which their performance is measured. We also examine span-level evaluation, where variation in annotation boundaries can substantially affect reported results, and propose Span Coverage F1 as a complementary containment-based metric. Our analysis clarifies what detector scores capture, highlights limitations of cross-benchmark comparisons, and motivates more explicit evaluation of generalization across benchmark conditions.

View arXiv page View PDF GitHub Add to collection

Models citing this paper 0

No model linking this paper

Cite arxiv.org/abs/2610.33608 in a model README.md to link it from this page.

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2610.33608 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2610.33608 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.

來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co