QDA:以 AI 智能體進行閉環大語言模型量化設計
QDA: Closed-Loop LLM Quantization Design with Agents
QDA以編碼智能體為不同模型及精度自主設計訓練後量化方案,無需人工介入。其方案在四個推理模型上,於 W4A16、W4A8、W3A16 分別平均領先最強公開方案 0.9、1.4、2.2 分,且可在現有 kernels 上執行而沒有推理開銷。對 320B 參數的 GLM-5.3-Flash 採用 NVFP4 時,6.5 小時、$69 的搜尋可恢復 FP8 模型 98.3% 的準確度。
QDA brings a coding agent into LLM quantization design. For each model and precision, the agent designs the post-training quantization recipe on its own, with no human in the loop. Its checkpoints beat the strongest public recipe on four reasoning models, by 0.9, 1.4 and 2.2 points on average at W4A16, W4A8 and W3A16, and run on existing kernels with no inference overhead.
The agent works the way a quantization expert does: it proposes a recipe, builds it, measures it and decides what to change next. Two things make this loop fast enough for up to 36 candidates in one eight-hour search: a KL score that takes minutes instead of dozens of GPU-hours, and a library of 21 PTQ methods that stack in any order. We find that no method wins everywhere: the best recipe differs for every model and precision, and the agent finds it by searching.
On the 320B-parameter GLM-5.3-Flash at NVFP4, one 6.5-hour, $69 search recovers 98.3% of the FP8 model's accuracy, against 96.9% for NVIDIA's release. The tools matter too: given NVIDIA ModelOpt's tools instead of ours, the same agent falls 0.5–1.3 points behind on Qwen3.6-35B. A search costs $19–75 of agent tokens.
來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co