跳到正文
HuggingFace Daily Papers(社區熱門論文)·· 2026-10-19AI 評分44

ScaleNav:用於語言條件式局部導航的可擴展網上影片預訓練

ScaleNav: Scalable Internet Video Pretraining for Language-Conditioned Local Navigation

AI 導讀

ScaleNav 從網上第一身視角影片學習目標導向的視覺動態,生成語言條件式局部導航所需的機械人動作。在 Short-Horizon OVON 單視角改編版及 HabitatGS-LN benchmark 上,模型分別取得 86.5% 和 94.9% 成功率,並勝過兩項 benchmark 上受評估的基線。ScaleNav 亦在真實四足機械人上部署及評估,結果支持其泛化至真實環境導航的能力。

正文

Published on Oct 19

Authors:

,

,

,

,

,

,

,

,

Abstract

ScaleNav learns goal-directed visual dynamics from internet egocentric videos to generate robot actions for language-conditioned local navigation, achieving high success rates in simulation and real-world deployment.

We study language-conditioned local navigation, where a robot must navigate to a text-specified target visible in its initial observation and stop at the described location. End-to-end navigation policies require diverse training data to generalize across scenes, yet real-robot data is costly to collect and simulated data leaves a sim-to-real gap. Internet egocentric videos offer a scalable source of real-world navigation experience. Our key insight is to learn goal-directed visual dynamics from Internet egocentric videos, and use these dynamics to support robot action generation. To this end, we propose ScaleNav, a unified world-action framework with three-stage training. We first perform instruction-conditioned video pretraining on these videos to learn how observations evolve as the camera approaches instructed targets across diverse real-world scenes. Action alignment then uses action-labeled simulated data to train the model to predict executable action chunks. Joint fine-tuning subsequently optimizes all trainable components together for navigation. ScaleNav achieves success rates of 86.5% on a single-view adaptation of Short-Horizon OVON and 94.9% on our HabitatGS-LN benchmark, outperforming the evaluated baselines on both benchmarks. We further deploy and evaluate ScaleNav on a real quadruped robot, with results supporting the model's ability to generalize to real-world navigation. Code and videos are available at https://zju3dv.github.io/ScaleNav/

View arXiv page View PDF Project page GitHub Add to collection

Get this paper in your agent:

hf papers read 2610.25690

Don't have the latest CLI?

curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 1

Yangyihui/ScaleNav

Robotics •

5B •

Updated 5 minutes ago

Datasets citing this paper 2

Yangyihui/ScaleNav-Data

Updated 11 minutes ago

•

97

Yangyihui/ScaleNav-OVON-Subset

Updated 11 minutes ago

•

15

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2610.25690 in a Space README.md to link it from this page.

Collections including this paper 1

來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co