受控基準測試釐清語言模型資料歸因方法的失效原因
Controlled benchmark clarifies failures in language-model data attribution
這項研究探討語言模型資料歸因方法失效的原因，指出現有評測往往同時改變多項數據集屬性。研究提出 CATT 受控事實追蹤基準，透過完全虛構的知識進行微調，並設定明確的真實答案。此類受控合成基準可用於評估研究進展，再進一步檢視真實且受多種因素幹擾的訓練數據。
The paper studies why language-model data-attribution methods fail, noting evaluations often vary several dataset properties at once. It introduces CATT, a controlled fact-tracing benchmark using fine-tuning on entirely fictitious knowledge with unambiguous ground truth.
Why it matters
Reliable attribution is needed to trace model outputs to training data, audit sources, and diagnose failures.
Analysis
Attribution research may adopt controlled synthetic benchmarks before claiming progress on real, confounded training data.
Source: arXiv
AI disclosure: this article was produced with AI assistance and reviewed for accuracy.
來源：DEV Community · dev.to