MM-ABC: Towards Generalist Mobile Manipulation via Seeing, Coordinating and Imagining
MM-ABC: Towards Generalist Mobile Manipulation via Seeing, Coordinating and Imagining
MM-ABC 是一個面向通用移動操作的基礎模型,結合空間感知、手臂與底盤協調,以及訓練期間使用的未來想像監督。在 EBench、RoboCasa365、LIBERO 和 LIBERO-Plus 分別取得 44.71%、61.2%、99.1% 和 82.8% 成功率,並在五項真實世界任務中取得 83% 平均成功率。模型以 5,000+ 小時、涵蓋 12 個數據集及 17 種機器人形態的數據預訓練。
Published on Sep 28
Authors:
,
,
,
,
,
,
,
Abstract
Mobile manipulation extends robot interaction beyond a fixed kinematic workspace by making the reachable region itself controllable. This flexibility introduces two central challenges: spatially grounded perception under continuous ego-motion and coordinated control of heterogeneous arm and base actions. Existing approaches strengthen geometry through explicit 3D representations or predictive world models, and often decouple mobility and manipulation into separate action streams. We argue that effective mobile manipulation requires not only decoupling, but also representations that support efficient cross-stream collaboration. We present MM-ABC, a foundation model built around Seeing, Coordinating, and Imagining Arm-Base Collaboration. MM-ABC combines sparse multi-level VLM features for spatial perception; a training-only future branch that uses world imagination and geometric intent as extra supervision, strengthening perception and manipulation-intent prediction and improving the overall learning signal; and MM-APT, which coordinates separate manipulation and mobility streams through masked joint attention and clean-action x-prediction. In controlled ablations, replacing clean-action prediction with velocity prediction lowers success on RoboCasa365 composite-seen tasks from 32.8% to 29.2%, and removing future supervision or multilevel conditioning causes larger drops. We pretrain MM-ABC on 5,000+ hours of heterogeneous robot data spanning 400K+ episodes, 12 datasets, and 17 embodiments. Experiments cover EBench, RoboCasa365, ManiSkill-HAB, LIBERO, LIBERO-Plus, and real-world mobile manipulation. MM-ABC achieves 44.71% success on EBench, 61.2% on RoboCasa365, 99.1% on LIBERO, 82.8% on LIBERO-Plus without perturbation training, and 83% mean success on five real-world tasks.
View arXiv page View PDF Project page Add to collection
Get this paper in your agent:
hf papers read 2609.35652
Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash
Models citing this paper 0
No model linking this paper
Cite arxiv.org/abs/2609.35652 in a model README.md to link it from this page.
Datasets citing this paper 1
Kivy/MM-30
selfcollect_mobile_dualarm • Updated about 4 hours ago • 2.44k episodes • 1.38k • 1
Spaces citing this paper 0
No Space linking this paper
Cite arxiv.org/abs/2609.35652 in a Space README.md to link it from this page.
Collections including this paper 0
No Collection including this paper
Add this paper to a collection to link it from this page.
來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co