FRED 以自蒸餾提升世界動作模型的未來影片表徵
Self-Distilling Future Video Representations for World Action Models
FRED透過自蒸餾及時間條件提升世界動作模型的單步未來影片表徵質素,改善長時程及未見任務的機械人策略表現。在只由人類影片示範、且未納入機械人訓練資料的任務上,FRED達到 50.0% 成功率,最佳競爭 WAM 為 19.4%。
Published on Oct 21
Authors:
,
,
,
,
Abstract
FRED improves single-step future video representations in world action models through self-distillation and temporal conditioning, boosting robot policy performance on long-horizon and unseen tasks.
World Action Models (WAMs) have emerged as a promising approach to learning generalist robot policies using rich spatiotemporal priors from pre-trained video diffusion models. A recent line of WAMs improves inference efficiency by conditioning a separate action decoder on intermediate future video representations extracted from a single denoising step, avoiding costly iterative denoising required for video generation. This design makes action prediction depend directly on the quality of these future representations. In this paper, we introduce Future REpresentation learning via self-Distillation with temporal context (FRED), a framework that trains WAMs to improve the quality of future video representations from noise in a single denoising step. Specifically, FRED uses self-distillation to predict future representations from noise, using representations extracted from ground-truth future observations as targets. We further condition the video model on historical observations, providing temporal context for predicting future representations better. Across diverse real-world manipulation scenarios, FRED consistently outperforms competing WAMs. It achieves particularly large gains on long-horizon and out-of-domain tasks under zero-shot environment generalization, and substantially improves human-to-robot skill transfer, reaching 50.0% task success on tasks absent from robot training data and demonstrated only in human videos, compared with 19.4% for the best competing WAM.
View arXiv page View PDF Project page GitHub Add to collection
Get this paper in your agent:
hf papers read 2610.30320
Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash
Models citing this paper 3
Robotics • Updated about 1 hour agoFRED-AGI-LAB/fred-wan22-5b-droid
Robotics • Updated about 1 hour agoFRED-AGI-LAB/fred-wan21-14b-droid
Robotics • Updated about 1 hour agoFRED-AGI-LAB/fred-wan22-5b-robocasa-kitchen
Datasets citing this paper 0
No dataset linking this paper
Cite arxiv.org/abs/2610.30320 in a dataset README.md to link it from this page.
Spaces citing this paper 0
No Space linking this paper
Cite arxiv.org/abs/2610.30320 in a Space README.md to link it from this page.
Collections including this paper 1
來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co