跳到正文
HuggingFace Daily Papers(社區熱門論文)·· 2026-10-09AI 評分41

SimVTA:以模擬為基礎的影片與軌跡對齊,應用於駕駛世界動作模型

SimVTA: Simulation-Grounded Video-Trajectory Alignment for Driving World Action Models

AI 導讀

SimVTA 結合模擬器渲染的未來影片監督與強化學習軌跡最佳化,提升駕駛世界動作模型的規劃表現。NAVSIM v1 上,對 WAM 初始化模型應用 SimVTA 後,PDMS 由 86.2 升至 91.3(+5.1);NAVSIM v2 得 88.2 EPDMS,HUGSIM 閉環評估的平均 HD-Score 為 38.8。推理時,模型直接預測軌跡,無須呼叫模擬器。

正文

Published on Oct 9

Authors:

,

,

,

,

,

,

,

,

,

Abstract

SimVTA improves driving world action models by combining simulator-rendered video supervision with reinforcement learning trajectory optimization.

Driving world action models (WAMs) learn from paired trajectories and future observations. Post-training with reinforcement learning allows the policy to explore alternative trajectories and improve planning using driving rewards. However, driving logs contain future videos only for the recorded trajectories, leaving these alternatives without corresponding visual supervision. Thus, we introduce SimVTA, a simulation-grounded post-training framework that combines trajectory optimization with video supervision. First, we sample candidate trajectories for each observation and use a real-world simulator (e.g., World Engine) to render future videos for high-reward candidates in real-world scenes reconstructed with 3D Gaussian Splatting (3DGS). This provides trajectory-matched supervision independent of the model's own predictions. Under a limited rendering budget, we prioritize video supervision for higher-error futures among these rendered candidates. Finally, we optimize trajectory planning with Group Relative Policy Optimization (GRPO) and use a flow-matching loss on the matched videos to update shared representations. At inference time, the model predicts trajectories directly, without simulator calls. On NAVSIM v1, applying SimVTA to a WAM initialization improves PDMS from 86.2 to 91.3 (+5.1). It achieves 88.2 EPDMS on NAVSIM v2 and an average HD-Score of 38.8 in closed-loop evaluation on HUGSIM. Ablations show that updating shared representations with video supervision improves planning, while prioritizing higher-error futures among high-reward trajectories further improves performance.

View arXiv page View PDF Project page GitHub Add to collection

Get this paper in your agent:

hf papers read 2610.13781

Don't have the latest CLI?

curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 0

No model linking this paper

Cite arxiv.org/abs/2610.13781 in a model README.md to link it from this page.

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2610.13781 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2610.13781 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.

來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co