訓練視覺模型以支援 LLM 驅動的權重空間程式設計
Training Vision Models for LLM-Driven Weight-Space Programming
研究團隊建立 LLM 驅動視覺模型權重空間程式設計基準,並發現專為編輯準備模型可大幅提升編輯成功率。研究以 5 個 LLM 在 4 個數據集上進行超過 2,600 次編輯;Edit-Ready Training(ERT)在 GPT-6 Astra 和 Claude Opus 5.5 的所有評測基線中取得最高平均分。團隊將公開測試框架、任務、編輯軌跡及評分。
Published on Nov 19
Authors:
Abstract
It has been demonstrated that, in certain settings, large language models can perform weight-space programming to directly modify the weights of neural networks. However, it is unknown how dedicated training of these networks can contribute to the successes of weight-space programming. We present a benchmark of LLM-driven weight-space programming on vision models, in which five LLMs edit convolutional networks and vision transformers on relabeling and error-correction tasks on Waterbirds, CIFAR-100, PACS and Entity-30. We report more than 2,600 such edit processes, whereby each edit is scored on held-out images for success, side effects and accuracy loss. We compare standard training with label smoothing, gradient-based editable training, and our Edit-Ready Training (ERT), a novel method that rehearses the direct, closed-form weight edits LLMs perform. We show that preparing networks for editing raises the success of LLM edits substantially where standard networks are hard to edit. ERT further achieves the highest mean score for GPT-6 Astra and Claude Opus 5.5 across all evaluated baselines. We will release our LLM testing framework alongside all tasks, editing trajectories and scores. Project page: https://viscom.uni-ulm.de/wsp
View arXiv page View PDF Add to collection
Get this paper in your agent:
hf papers read 2611.26408
Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash
Models citing this paper 0
No model linking this paper
Cite arxiv.org/abs/2611.26408 in a model README.md to link it from this page.
Datasets citing this paper 0
No dataset linking this paper
Cite arxiv.org/abs/2611.26408 in a dataset README.md to link it from this page.
Spaces citing this paper 1
Collections including this paper 0
No Collection including this paper
Add this paper to a collection to link it from this page.
來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co