跳到正文
HuggingFace Daily Papers(社區熱門論文)·· 12 天前AI 評分41

DexForesight:以隱式空間前瞻能力為靈巧操作策略提供空間基礎

DexForesight: Grounding Dexterous Policies with Implicit Spatial Foresight

AI 導讀

DexForesight 透過把具備動態感知能力的世界表徵注入靈巧操作策略的潛在空間,讓策略隱式學習空間前瞻能力。在 11 項 DexJoCo 模擬任務中,基礎策略的整體成功率由 49.4% 提升至 60.5%;在 3 項真實機械人任務中,成功率為 58.3%,高於最強基線的 50.0%。

正文

Published on Nov 10

Authors:

,

,

,

,

,

,

,

Abstract

Dexterous manipulation requires rich spatial foresight over both the surrounding scene and the robot hand. Existing world-action models often acquire such predictive knowledge through explicit future prediction. We introduce DexForesight, a framework that learns spatial foresight implicitly by injecting dynamics-aware world representations into the latent space of a dexterous policy. During training, DexForesight learns an action-conditioned spatial world model from offline scene depth, kinematic hand geometry, and demonstration trajectories. The world model predicts 3D hand-point motion and visual feature changes, capturing both articulated hand dynamics and scene changes under demonstrated actions, including the motion of fingers occluded from the cameras. We then freeze the world model and align policy queries with its predictive world queries, transferring the learned spatial dynamics into policy latents and using them to condition action generation. At deployment, only the policy is retained, operating on RGB observations, language, and robot state without depth input or explicit future prediction. On 11 DexJoCo simulation tasks, DexForesight improves the overall success rate of the base policy from 49.4% to 60.5%. On three real-robot dexterous manipulation tasks, DexForesight achieves 58.3% success, compared with 50.0% for the strongest baseline. Project page https://dexforesight.github.io/

View arXiv page View PDF Project page Add to collection

Get this paper in your agent:

hf papers read 2611.11166

Don't have the latest CLI?

curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 0

No model linking this paper

Cite arxiv.org/abs/2611.11166 in a model README.md to link it from this page.

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2611.11166 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2611.11166 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.

來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co