到手即失效:公開模型庫中失效量化大語言模型檔案的檢測方法
Broken on Arrival: Non-Functional Quantized LLM Artifacts in Public Model Registries and How to Catch Them
這項研究檢測公開模型庫中的量化大語言模型檔案,發現部分低位元量化版本無法正常運作。研究測試了 Ollama 官方庫 15 個模型系列的 305 個符合條件、具程式碼能力的 GGUF 標籤,以及 22 個熱門 Hugging Face 社羣檔案;7 個官方標籤(2.3%)在 164 項 HumanEval+ 任務中最多解出 6 項。
Published on Oct 25
Authors:
Abstract
Quantized small language model files in public registries can be non-functional due to uncalibrated quantization recipes, and screening plus calibration is recommended before release.
Developers download quantized large language models (LLMs) from public registries to run them locally. None of the registries we examined documented functional testing of their files. We tested all 305 eligible code-capable GGUF tags (the llama.cpp format) across 15 model lines in the official Ollama library, plus 22 popular Hugging Face community files. A 15-task code screen identified suspects, which we retested on all 164 HumanEval+ tasks. Seven official tags (2.3%) solved at most six of them. All were low-bit quantizations of models with at most four billion parameters. We rebuilt five suspect files from the original weights. All five registry files matched our uncalibrated rebuilds tensor for tensor, so none was corrupted. Exact reproducibility did not imply function. With the weights, toolchain and tensor mix held fixed, adding an importance matrix raised three Qwen2.5-Coder-3B builds from zero to 76-113 solved tasks. The same uncalibrated recipe left the smaller Qwen2.5-Coder-1.5B functional, showing that the recipe alone does not predict failure. None of 66 sampled tags that passed the screen was non-functional. We release the audit data and the quantcheck tool and recommend screening low-bit tags before publication, then calibrating or withholding failures.
View arXiv page View PDF Project page GitHub 0 Add to collection
Get this paper in your agent:
hf papers read 2609.05881
Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash
Models citing this paper 0
No model linking this paper
Cite arxiv.org/abs/2609.05881 in a model README.md to link it from this page.
Datasets citing this paper 1
aditiPatodiya/quantcheck-audit
Viewer • Updated 1 day ago • 9.66k
Spaces citing this paper 0
No Space linking this paper
Cite arxiv.org/abs/2609.05881 in a Space README.md to link it from this page.
Collections including this paper 0
No Collection including this paper
Add this paper to a collection to link it from this page.
來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co