跳到正文
HuggingFace Daily Papers(社區熱門論文)·· 3 天前AI 評分45

學習動作如何改變結果:以動作效應匹配提升世界模型的動態保真度

Learning What Actions Change: Action Effect Matching for Dynamics-Faithful World Models

AI 導讀

AFFECT 透過訓練世界模型匹配時間相關動作噪聲引發的視覺變化,提升模型推演結果與真實環境的吻合度。在 DROID 上,模型中與真實機械人成功率的 Pearson 相關係數由 0.47 升至 0.79,FVD 最多降低 23%。使用經 AFFECT 訓練的世界模型選擇動作,任務成功率最多提升 8.3%。

正文

Published on Nov 19

Authors:

,

Abstract

AFFECT, an auxiliary objective that trains world models to match visual responses to temporally correlated action noise, improves rollout fidelity and planning success in simulated and real robotic tasks.

World models aim to predict environment dynamics under actions, enabling the prediction of action consequences without executing them in the real world. Yet despite producing increasingly realistic videos, current world models often fail to capture how small action changes affect future observations. This limits their use in tasks such as robotic policy evaluation and model-based planning, where nearly identical actions can lead to qualitatively different outcomes. To address this, we propose AFFECT, an auxiliary training objective for learning the distribution of visual changes induced by action perturbations. Specifically, we inject temporally correlated noise into the actions of a policy trained on task-agnostic human teleoperation. We then train the world model so that its generated videos respond to this noise as the recorded videos do, without requiring repeated executions from the same initial state. Across OGBench simulation and real-world DROID and ALOHA-2 setups, AFFECT makes world-model rollouts more faithful to real outcomes. On DROID, the Pearson correlation between policy success rates in the world model and on the real robot rises from 0.47 to 0.79, alongside up to 23% lower FVD. These gains carry over to planning, where selecting actions with AFFECT-trained world models improves task success by up to 8.3%.

View arXiv page View PDF Project page Add to collection

Get this paper in your agent:

hf papers read 2611.25980

Don't have the latest CLI?

curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 0

No model linking this paper

Cite arxiv.org/abs/2611.25980 in a model README.md to link it from this page.

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2611.25980 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2611.25980 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.

來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co