跳到正文
HuggingFace Daily Papers(社區熱門論文)·· 2026-10-13AI 評分47

Training LLMs for Multi-Agent Collaboration with Verifiable Information Rewards

Training LLMs for Multi-Agent Collaboration with Verifiable Information Rewards

AI 導讀

CoSIR 以可驗證資訊獎勵和自我對弈強化學習微調 LLM,將協作能力納入訓練目標。在 7 項未見過的自然語言協作基準測試中,CoSIR 在 Qwen3-4B、Qwen3-8B 和 Gemma-3-4B-it 三個模型上均取得最高總分;Qwen3-8B 比基礎模型高 31.85 分。模型在 GSM8K、MMLU 和 IFEval 的表現大致維持。

正文

🚀 Can we train LLMs to collaborate, rather than only design better multi-agent workflows?

I'm sharing our new preprint, Training LLMs for Multi-Agent Collaboration with Verifiable Information Rewards (CoSIR).

Most LLM post-training focuses on chatbots and individual agents: instruction following, reasoning, and task completion. In multi-agent systems (MAS), however, connecting several LLMs through prompts or workflows does not guarantee effective teamwork.

Agents need to learn when to share useful evidence, when to ask teammates for missing information, and how to incorporate their contributions into decisions.

So we ask: can collaboration itself be an explicit target of LLM post-training?

💡 The challenge is reward design. In natural-language tasks, final success tells us little about which messages helped. A statement's value depends on what teammates already know and whether it helps them identify the correct answer.

We introduce CoSIR with three components:

🔹 Scalable, verifiable environment. Agents see different parts of a rating table and must exchange information to select the best option. We can generate many tasks while tracking each agent's information and the correct answer.

🔹 Verifiable information rewards. A Bayesian evaluator scores shared evidence by its reduction in team surprisal. Requests are scored by the potential value of truthful answers, with additional rewards for correctly using received information.

🔹 Self-play RL. These fine-grained rewards directly post-train the underlying LLM, rather than only modifying an inference-time workflow.

📊 The key test is transfer to more complex tasks—not performance on the toy task alone.

We post-train Qwen3-4B, Qwen3-8B, and Gemma-3-4B-it, then evaluate them on 7 unseen natural-language collaboration benchmarks spanning distributed evidence, mathematical reasoning, and collaborative decision-making. CoSIR achieves the highest seven-task score sum among compared methods on all 3 backbones (Qwen3-8B: +31.85 points versus base).

We also examine behavior. For Qwen3-8B on HiddenBench, private-fact disclosure rises from 68.5% to 74.3%. In MAST-based analysis pooled across the three models, failures to use teammates' input fall from 26.3% to 22.8%, and failures to seek clarification from 36.4% to 32.0%. Performance on GSM8K, MMLU, and IFEval remains largely preserved.

🎯 Our takeaway: Collaboration is a capability that should be explicitly trained. Better workflows matter—but so does teaching the underlying LLM how to work with other agents.

📄 https://arxiv.org/abs/2610.17893

來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co