跳到正文
HuggingFace Daily Papers(社區熱門論文)·· 7 天前AI 評分45

開放權重大語言模型技術報告的新穎性檢視:消融研究、可重現性與透明度

The novelty exposed: ablation, reproducibility and transparency in 40 technical reports of open-weight large language models (LLMs)

AI 導讀

研究檢視 40 份開放權重大語言模型技術報告,評估其新穎性主張是否有消融研究支持,以及報告的可重現性與透明度。73 項架構或訓練新穎性主張中,19 項有受控對照;40 份報告中,4 份披露多於一次訓練,只有 1 份同時披露數據混合、token 數量及計算資源。研究提出 7 項技術報告撰寫規則,並指出不評判模型質素,未經測試的主張仍可能成立。

正文

Published on Nov 15

Authors:

Abstract

In 2025 alone, 48 new large language model (LLM) architectures were added to the Hugging Face transformers library, and many of them arrive with a technical report that presents the model as new. What is actually new in these models, and is it tested? We put this question to LLM technical reports. We look at three things: what the reference code shows about how new open-weight architectures differ from earlier ones; what 40 randomly sampled technical reports claim as new and whether they support it with ablation studies; and what they disclose for reproducibility and transparency. In the library, 39 of the 48 architectures added in 2025 take at least two of their three core blocks from another model, most often Llama, although part of this reflects how the library writes code. In the reports, 19 of 73 architectural or training novelty claims come with a controlled with/without comparison; 4 of 40 reports state more than one training run; 8 rerun the models they compare against under one setup; and 1 discloses data mixture, token count, and compute together. We also looked for known components given new names: none of five candidate cases hid its origin, though we did not study model names. We propose seven reporting rules for LLM release reports and apply them to two reports. We do not judge model quality, and an untested claim may still be true.

View arXiv page View PDF Project page GitHub 0 Add to collection

Get this paper in your agent:

hf papers read 2611.20003

Don't have the latest CLI?

curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 0

No model linking this paper

Cite arxiv.org/abs/2611.20003 in a model README.md to link it from this page.

Datasets citing this paper 1

bturkoglu/llm-novelty-exposed

Viewer •

Updated about 9 hours ago

•

284

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2611.20003 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.

來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co