跳到正文
HuggingFace Daily Papers(社區熱門論文)·· 3 天前AI 評分38

AutoMaster:透過讓同一模型成為更好的教師實現自主後訓練

AutoMaster: Autonomous Post-Training by Making the Same Model a Better Teacher

AI 導讀

AutoMaster 是一套自主後訓練系統,透過分析軌跡、設計技能、篩選資料及評估目標模型,改善同一模型生成教師軌跡的方式。系統在 Terminal、Search 和軟件工程任務中,以 4B 和 9B 目標模型測試,較 Vanilla SFT 的平均表現提升 1.42-8.99 個百分點。六種設定均顯示,微調時保留技能指示的平均表現較移除指示更高,即使評估時沒有這些指示,監督輸出亦相同。

正文

Published on Nov 19

Authors:

,

,

,

,

Abstract

Task success alone does not specify which behaviors post-training should reinforce. Even successful teacher trajectories can contain unsupported assumptions, unproductive repetition, or superficial verification. We propose Skill-FT (Skill-Guided Fine-Tuning), which uses natural-language skills to suppress undesirable habits and elicit underrepresented effective behaviors during teacher rollouts, then fine-tunes a target model on the resulting trajectories. We introduce AutoMaster, an autonomous post-training system in which an LLM agent develops these interventions through trajectory analysis, skill design, data selection, and feedback from rollouts and target-model evaluation. Across Terminal, Search, and software engineering tasks with 4B and 9B target models, AutoMaster improves mean performance over Vanilla SFT on successful unguided teacher trajectories by 1.42-8.99 percentage points, with the teacher, task pool, training set size, and SFT recipe held fixed. Together with the ablations and trajectory analyses, these results suggest that behavioral shaping can make a fixed model a more effective teacher. We further find that retaining skill instructions during SFT yields higher mean performance than removing them across all six settings, despite their absence at evaluation and identical supervised outputs. These findings suggest that preserving skill context as part of a training demonstration can support better behavioral internalization.

View arXiv page View PDF Project page GitHub 3 Add to collection

Get this paper in your agent:

hf papers read 2611.26510

Don't have the latest CLI?

curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 0

No model linking this paper

Cite arxiv.org/abs/2611.26510 in a model README.md to link it from this page.

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2611.26510 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2611.26510 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.

來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co