社交智能體的互動如何才有效?從成對情境的可驗證結果中學習
What Makes a Social Agent's Interactions Effective? Learning from Verifiable Outcomes in Paired Worlds
研究提出 HICO-Bench,以 180 個日常協調情境和可驗證結果評估社交智能體的互動成效。PaCPO 根據每輪暫定計劃在兩個情境版本中的適配情況給予獎勵,使 Qwen3-8B 的已驗證成功率由 20.4% 升至 28.3%。在 13 個 LLM 的評估中,獲高度目標完成評分的對話有 36% 未達已驗證成功。
Published on Oct 14
Authors:
,
,
,
,
,
,
Abstract
Effective social interaction requires more than fluent responses: agents must reach plans that fit the participants' needs and circumstances. Existing evaluations of social agents mainly rely on LLM judges that rate the conversation, without checking the resulting plan. We introduce HICO-Bench, a benchmark of 180 everyday coordination scenarios, such as resource allocation, scheduling, and authorization, in which two agents with private information converse and then independently record the plan they agreed on. A program checks Feasibility (the two records match and satisfy both participants' private conditions), and an LLM judge checks Integrity (the conversation supports the recorded plan); passing both is a Verified Success. Each scenario has two versions that share the same public context but differ in one private condition, so the correct plan differs between them. Across thirteen LLMs, 36% of conversations rated highly for goal completion fail Verified Success. To learn from outcomes that arrive only at the end of a conversation, we propose Paired-Credit Policy Optimization (PaCPO). After each turn, PaCPO asks the agent for a tentative plan, checks it in both versions of the scenario, and rewards the turns after which the tentative plans fit their respective versions. PaCPO raises Qwen3-8B's Verified Success from 20.4% to 28.3%.
View arXiv page View PDF Add to collection
Get this paper in your agent:
hf papers read 2610.20195
Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash
Models citing this paper 0
No model linking this paper
Cite arxiv.org/abs/2610.20195 in a model README.md to link it from this page.
Datasets citing this paper 0
No dataset linking this paper
Cite arxiv.org/abs/2610.20195 in a dataset README.md to link it from this page.
Spaces citing this paper 0
No Space linking this paper
Cite arxiv.org/abs/2610.20195 in a Space README.md to link it from this page.
Collections including this paper 0
No Collection including this paper
Add this paper to a collection to link it from this page.
來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co