跳到正文
HuggingFace Daily Papers(社區熱門論文)·· 2026-10-20AI 評分37

MultiAgent-OPD:以 On-Policy Distillation 端到端訓練階層式多智能體系統

MultiAgent-OPD: End-to-End Training of Hierarchical Multi-Agent Systems with On-Policy Distillation

AI 導讀

MultiAgent-OPD 以 On-Policy Distillation 端到端訓練階層式多智能體系統,結合 Searcher 的局部過程監督,以及 Leader 和 Answerer 基於結果的強化學習。

正文

✨ Highlights

Role-specific learning signals — MultiAgent-OPD combines local process supervision for Searchers with outcome-based reinforcement learning for the Leader and Answerer, addressing credit assignment in hierarchical multi-agent teams.

Two-stage training recipe — First trains a teacher to search and summarize in a cascaded Search–Answer workflow, then transfers its expertise to Searchers during joint hierarchical training.

Token-level on-policy distillation — A frozen, task-trained teacher scores student-generated queries and summaries on evolving subtask trajectories, providing fine-grained guidance for evidence acquisition and reporting.

Joint training with a shared model — Leader, Searcher, and Answerer roles jointly optimize a shared language model to coordinate task decomposition, multi-wave search, and answer generation. Inference requires only the trained student.

Strong performance across seven QA benchmarks — MultiAgent-OPD achieves 38.68% macro avg@4, outperforming M-GRPO like LLM Judge MAS-RL method by 4.06 percentage points and C-MAS + RL (cascaded MAS-RL method) by 2.61 points at the evaluated checkpoints.

來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co