跳到正文
HuggingFace Daily Papers(社區熱門論文)·· 13 天前AI 評分56

World In-Context Learning 用於智能體機械人操作

World In-Context Learning for Agentic Robot Manipulation

AI 導讀

研究提出 World In-Context Learning(WICL)及 RoboScript,讓凍結模型在模擬環境排演示範任務,並把回饋寫入上下文而非模型權重。在 RoboTwin 2.0 全部 50 項任務中,每項只用一個示範且不更新梯度,平均成功率達 83.4%,比最強訓練式動作策略高逾 20 個百分點。RoboScript 亦透過跨具身記憶適配,將方法遷移至實體雙臂機械人。

正文

Abstract

Trained action policies write manipulation skill into network weights, tying what is learned to the distribution it is learned from. In-context learning (ICL) avoids this coupling but has not matched trained policies, because a demonstration recorded in another scene carries no feedback from the world the robot currently faces. We observe that what an agent learns in context is semantic, recording task steps and object roles rather than visual appearance, so experience gathered in one world transfers to another. Based on this insight, we propose World In-Context Learning (WICL), a paradigm in which a frozen model starts from a demonstration, rehearses the task in a world such as a simulator, and writes the feedback into its context rather than its weights. We build RoboScript, a modular infrastructure that implements WICL for agentic manipulation. From a single demonstration, RoboScript learns each task through rehearsal in simulation, freezes the learned knowledge as a Task Memory, and deploys the frozen memory on new scenes with data isolation that ensures every deployment input is available on a real robot. On all 50 tasks of RoboTwin 2.0, RoboScript reaches 83.4% mean success across clean and randomized settings with one demonstration per task and no gradient updates, exceeding the strongest trained action policy by over 20 points. The method generalizes across visual distributions, improves through reflection, scales with the foundation model, and transfers to a physical dual-arm robot through cross-embodiment memory adaptation. With a world in the loop, a robot can learn a manipulation task in its context as well as it learns one in its weights. All code is available at https://github.com/ArcheOne-AI/RoboScript

View arXiv page View PDF Project page GitHub 36 Add to collection

Models citing this paper 0

No model linking this paper

Cite arxiv.org/abs/2611.12885 in a model README.md to link it from this page.

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2611.12885 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2611.12885 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.

來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co