跳到正文
HuggingFace Daily Papers(社區熱門論文)·· 5 天前AI 評分44

TCP4D:用於機械人操作的統一工具中心點追蹤與生成模型

TCP4D: A Unified Tool-Center-Point Tracking and Generation Model for Robot Manipulation

AI 導讀

TCP4D以共享的4D場景表示追蹤觀測到的TCP運動,並生成未來TCP軌跡,是面向機械人操作的統一模型。追蹤器在DROID及RoboLab的絕對軌跡誤差分別為7.60及16.07 mm,僅用十個示範即可適應新機械人形態。生成器在RoboTwin的平均成功率為69.19%;聯合訓練源機械人及增強示範,較僅用增強示範訓練提升目標機械人成功率41.5%。

正文

Published on Nov 17

Authors:

,

,

,

,

,

,

Abstract

Robot manipulation videos provide a rich visual experience, yet recovering robot action and gripper-state annotations from RGB observations still remains challenging. Using the embodiment-agnostic tool-center-point (TCP) pose as the action space, we present TCP4D, a unified model in which a shared 4D scene representation both tracks observed TCP motion and generates future TCP trajectories. Given a monocular video and initial 2D TCP queries, the tracker directly recovers metric TCP poses in SE(3) and gripper states alongside scene geometry and camera parameters. The generator samples local features from the same 4D representation along the recovered TCP tracks, then combines them with robot observations and language instructions to predict future TCP states. Our new cross-embodiment augmentation pipeline converts existing RGB videos into demonstrations for alternative embodiments by retargeting the recovered motion and synthesizing the corresponding observations, without recorded source-robot states. Experiments show that the tracker achieves absolute trajectory errors of 7.60 and 16.07 mm on DROID and RoboLab and adapts to a new embodiment with only ten demonstrations. The generator achieves 69.19% mean success on RoboTwin and outperforms well-established policy models on two real-world tasks, while joint training on source and augmented demonstrations improves target-robot success by 41.5% over augmented-only training.

View arXiv page View PDF Project page GitHub Add to collection

Get this paper in your agent:

hf papers read 2611.23143

Don't have the latest CLI?

curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 0

No model linking this paper

Cite arxiv.org/abs/2611.23143 in a model README.md to link it from this page.

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2611.23143 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2611.23143 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.

來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co