SkeLAM：以骨架為基礎的機器人世界模型潛在動作模型
Skeleton-Grounded Latent Action Model for Robot World Models
SkeLAM 將潛在動作分解為由機器人骨架監督的運動學成分，以及保留其他視覺動態的殘差成分，以提升機器人世界模型的可控性及動作遷移能力。在 DROID 上，相較 DreamDojo，SkeLAM 將末端執行器 ADE 降低 19.5%，LPIPS 降低 11.1%。生成結果展示零樣本人類至機器人動作遷移，政策評估中預測成功率亦與真實機器人執行結果一致。
Published on Oct 22
Authors:
,
,
,
,
Abstract
SkeLAM decomposes latent actions into skeleton-guided kinematic and residual components to improve controllable robot world modeling and motion transfer.
Robot world models simulate how environments evolve in response to robot actions. Latent action models enable learning dynamics from videos by encoding visual transitions as proxy actions. However, reconstruction-driven latent actions mix robot motion with object responses and other scene changes, complicating their correspondence with robot controls. We propose SkeLAM, a skeleton-guided latent action model for controllable robot world modeling. SkeLAM decomposes latent actions into a kinematic component grounded by robot skeleton supervision and a residual component retaining complementary visual dynamics. Cross-modal swap reconstruction encourages RGB-derived and skeleton-derived kinematic representations to be interchangeable during video reconstruction. The world model incorporates both components through separate conditioning pathways, while branch dropout reduces its dependence on future-derived residual information. A lightweight adapter maps robot actions to the kinematic space, and subsequent world-model fine-tuning adapts generation to these predicted conditions. At inference, the model generates future observations from the current observation and robot actions. On DROID, SkeLAM reduces end-effector ADE by 19.5% and LPIPS by 11.1% compared with DreamDojo. Motion probing supports kinematic specialization, while qualitative results demonstrate zero-shot human-to-robot motion transfer in generated rollouts. Policy evaluation shows agreement between predicted success rates and those measured in real robot executions.
View arXiv page View PDF Project page Add to collection
Get this paper in your agent:
hf papers read 2610.31474
Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash
Models citing this paper 0
No model linking this paper
Cite arxiv.org/abs/2610.31474 in a model README.md to link it from this page.
Datasets citing this paper 0
No dataset linking this paper
Cite arxiv.org/abs/2610.31474 in a dataset README.md to link it from this page.
Spaces citing this paper 0
No Space linking this paper
Cite arxiv.org/abs/2610.31474 in a Space README.md to link it from this page.
Collections including this paper 0
No Collection including this paper
Add this paper to a collection to link it from this page.
來源：HuggingFace Daily Papers（社區熱門論文） · huggingface.co