跳到正文
HuggingFace Daily Papers(社區熱門論文)·· 2026-09-30AI 評分45

ReSAIL: Mitigating Collapse in Iterative Agent Self-Distillation

ReSAIL: Mitigating Collapse in Iterative Agent Self-Distillation

AI 導讀

ReSAIL 以軌跡平衡選擇性蒸餾和特權資訊保留兩項組件,緩解迭代式智能體自我蒸餾中的崩塌。使用 Qwen3-4B 和 Qwen3-8B,在 ALFWorld 和 TextCraft 上連續三個部署週期維持效能提升;第三個週期的成功率平均較相應 SDPO/OEL 基線高 22.5 個百分點。程式碼、訓練及評估腳本和模型檢查點均已公開。

正文

Can agents keep improving by distilling from their own deployment experience? In our experiments, existing self-distillation methods can lose both deployment performance and task competence with privileged information (PI) across cycles. This matters because today's student becomes tomorrow's teacher.

ReSAIL is a plug-in augmentation with two complementary components:

  • Trajectory-balanced selective distillation: prioritize interaction steps where PI most changes the teacher's predictions, and balance the losses across trajectories.
  • Privileged retention: preserve the student's PI-conditioned behavior to support supervision in the next cycle.

With Qwen3-4B and Qwen3-8B, ReSAIL sustains gains over three deployment cycles on ALFWorld and TextCraft. At cycle 3, it improves success rates by 22.5 percentage points on average over the corresponding SDPO/OEL baselines, across two model scales and three evaluation settings (ALFWorld ID/OOD and TextCraft).

Code, training/evaluation scripts, and model checkpoints are publicly available. Happy to discuss the method, results, and limitations!

來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co