跳到正文
HuggingFace Daily Papers(社區熱門論文)·· 2026-10-15AI 評分37

Mosaic:多工作階段智能體強化學習

Mosaic: Multi-Session Agentic Reinforcement Learning

AI 導讀

Mosaic 提出多工作階段智能體強化學習方法,透過不使用 critic 的階層式最佳化,按各工作階段的邊際貢獻調整優勢估計。在 BrowseComp 上以 Qwen3.5-4B 進行的實驗中,Mosaic 在上下文壓縮及子智能體分叉情境均勝過把軌跡層級優勢廣播至所有工作階段的方法,平均使用的工作階段亦較少。

正文

Published on Oct 15

Authors:

,

,

,

,

Abstract

Multi-session agentic reinforcement learning is introduced to handle trajectories across multiple sessions, with Mosaic optimizing session-level contributions and allocation via hierarchical advantage estimation.

Most existing agentic reinforcement learning methods assume single-session trajectories, where interaction proceeds within one continuously accumulated context. However, agentic techniques such as context compaction and subagent forking may produce trajectories spanning multiple sessions, creating a mismatch between the single-session training assumption and inference-time execution. Therefore, we introduce Multi-Session Agentic Reinforcement Learning, a more general setting that supports an arbitrary number of sessions within a trajectory. Under this setting, a widely adopted method is to broadcast trajectory-level advantages to all sessions, but this does not distinguish contributions from various sessions or guide session allocation within trajectories. To address this issue, we propose Mosaic, a multi-session agentic RL method with critic-free hierarchical optimization. At the trajectory level, it incorporates session-usage signals into advantage estimation. At the session level, it shapes each session's advantage according to the estimated marginal contribution. Experiments on BrowseComp with Qwen3.5-4B show that Mosaic outperforms broadcasting in both context compaction and subagent forking scenarios while using fewer sessions on average. Further analyses show that our method achieves improvements in session utility and allocation, which we suggested is a promising target for future advances in multi-session agentic RL.

View arXiv page View PDF Project page GitHub Add to collection

Get this paper in your agent:

hf papers read 2610.22174

Don't have the latest CLI?

curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 0

No model linking this paper

Cite arxiv.org/abs/2610.22174 in a model README.md to link it from this page.

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2610.22174 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2610.22174 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.

來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co