跳到正文
HuggingFace Daily Papers(社區熱門論文)·· 2026-10-09AI 評分44

WATCH:從任務配對的人類影片學習可遷移的世界動作模型

Learning Transferable World-Action Models from Task-Paired Human Videos

AI 導讀

WATCH 透過跨影片預訓練,從不同環境的任務配對人類示範中學習可遷移的世界動作表示,無需配對的人類與機械人示範。機械人後訓練後,MimicDroid 配合 Cosmos Policy 的情境內成功率由 40.9% 升至 68.3%,配合 Cosmos3 則由 59.8% 升至 77.7%。提升在未見關卡最大,並在跨機械人形態的無上下文後訓練中持續。

正文

Published on Oct 9

Authors:

,

,

Abstract

World-action models inherit broad visual and physical priors from video pre-training. However, standard within-video next-frame prediction allows them to rely on appearance shared between the context and future, rather than learning interactions that transfer across scenes and embodiments. We introduce WATCH, a cross-video pre-training that learns from task-paired human demonstrations collected across diverse environments. Conditioned on the video and actions of one demonstration, the model jointly predicts another demonstration's actions and future frames from its current observation. Predicting one demonstration from another encourages the model to learn task-relevant interactions that transfer across scenes and demonstrators. Our pre-training shows that these capabilities can be learned from readily scalable human-human pairs alone, without requiring matched human-robot demonstrations. Analyses before robot post-training show improved motion alignment across embodiments and better action and video prediction on unseen robots, with representations closer to those learned from robot pairs. After robot post-training, WATCH improves MimicDroid in-context success from 40.9% to 68.3% with Cosmos Policy and from 59.8% to 77.7% with Cosmos3, with the largest gains on the unseen levels. Gains persist under context-free post-training across robot embodiments. WATCH outperforms within-video pre-training and co-training baselines, supporting cross-video prediction as a scalable approach to learning transferable world-action representations.

View arXiv page View PDF Add to collection

Get this paper in your agent:

hf papers read 2610.14351

Don't have the latest CLI?

curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 0

No model linking this paper

Cite arxiv.org/abs/2610.14351 in a model README.md to link it from this page.

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2610.14351 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2610.14351 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.

來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co