ScaleNav:用於語言條件式局部導航的可擴展網上影片預訓練
ScaleNav: Scalable Internet Video Pretraining for Language-Conditioned Local Navigation
ScaleNav 從網上第一身視角影片學習目標導向的視覺動態,生成語言條件式局部導航所需的機械人動作。在 Short-Horizon OVON 單視角改編版及 HabitatGS-LN benchmark 上,模型分別取得 86.5% 和 94.9% 成功率,並勝過兩項 benchmark 上受評估的基線。ScaleNav 亦在真實四足機械人上部署及評估,結果支持其泛化至真實環境導航的能力。
Published on Oct 19
Authors:
,
,
,
,
,
,
,
,
Abstract
ScaleNav learns goal-directed visual dynamics from internet egocentric videos to generate robot actions for language-conditioned local navigation, achieving high success rates in simulation and real-world deployment.
We study language-conditioned local navigation, where a robot must navigate to a text-specified target visible in its initial observation and stop at the described location. End-to-end navigation policies require diverse training data to generalize across scenes, yet real-robot data is costly to collect and simulated data leaves a sim-to-real gap. Internet egocentric videos offer a scalable source of real-world navigation experience. Our key insight is to learn goal-directed visual dynamics from Internet egocentric videos, and use these dynamics to support robot action generation. To this end, we propose ScaleNav, a unified world-action framework with three-stage training. We first perform instruction-conditioned video pretraining on these videos to learn how observations evolve as the camera approaches instructed targets across diverse real-world scenes. Action alignment then uses action-labeled simulated data to train the model to predict executable action chunks. Joint fine-tuning subsequently optimizes all trainable components together for navigation. ScaleNav achieves success rates of 86.5% on a single-view adaptation of Short-Horizon OVON and 94.9% on our HabitatGS-LN benchmark, outperforming the evaluated baselines on both benchmarks. We further deploy and evaluate ScaleNav on a real quadruped robot, with results supporting the model's ability to generalize to real-world navigation. Code and videos are available at https://zju3dv.github.io/ScaleNav/
View arXiv page View PDF Project page GitHub Add to collection
Get this paper in your agent:
hf papers read 2610.25690
Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash
Models citing this paper 1
Robotics • 5B • Updated 5 minutes agoYangyihui/ScaleNav
Datasets citing this paper 2
Yangyihui/ScaleNav-Data
Yangyihui/ScaleNav-OVON-Subset
Spaces citing this paper 0
No Space linking this paper
Cite arxiv.org/abs/2610.25690 in a Space README.md to link it from this page.
Collections including this paper 1
來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co