Rethinking World-Action Model for Compositional and In-Context Robotic Manipulation
Rethinking World-Action Model for Compositional and In-Context Robotic Manipulation
ViGAR 提出分層框架,將機械人操作拆分為視覺子目標規劃和子目標執行,以支援長時程組合式操作及情境內學習。在 RoboTwin Clean2Random benchmark 的 Clean 和 Random 設定下,ViGAR 的成功率分別為 82.00% 和 67.02%,平均成功率較最強基線高 12.86 個百分點。五項組合式任務及兩項情境內學習任務的真實機械人實驗亦驗證了框架的效果。
Published on Oct 1
·
on Oct 8
Authors:
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
Abstract
Long-horizon compositional manipulation has become increasingly important for real-world robot deployment, where a single task involves multiple coordinated subtasks. Existing world-action models (WAMs) jointly predict short-horizon visual futures and actions, but typically lack explicit subtask-level reasoning. We propose Visual Goal-conditioned Action Reasoning (ViGAR), a hierarchical framework that factorizes manipulation into a visual subgoal planner and a subgoal executor. Given the current observation and global instruction, the subgoal planner predicts a visual subgoal for the next subtask. The subgoal executor then jointly generates future visual trajectories and actions conditioned on the predicted subgoal. Both components share a pretrained world-model representation, enabling task-level planning and action generation to benefit from common physical knowledge. Moreover, our framework naturally supports in-context learning: using a global goal image as context can induce different subtask decompositions and behaviors without parameter updates. On the RoboTwin Clean2Random benchmark, ViGAR achieves 82.00% and 67.02% success rates under the Clean and Random settings, respectively, surpassing the strongest baseline by 12.86 percentage points in average success rate. Real-world robot experiments on five compositional and two in-context learning tasks further confirm the effectiveness of ViGAR.
View arXiv page View PDF Add to collection
Get this paper in your agent:
hf papers read 2610.02368
Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash
Models citing this paper 1
Robotics • Updated about 13 hours agoDAGroup-PKU/ViGAR
Datasets citing this paper 0
No dataset linking this paper
Cite arxiv.org/abs/2610.02368 in a dataset README.md to link it from this page.
Spaces citing this paper 0
No Space linking this paper
Cite arxiv.org/abs/2610.02368 in a Space README.md to link it from this page.
Collections including this paper 0
No Collection including this paper
Add this paper to a collection to link it from this page.
來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co