HuggingFace Daily Papers(社區熱門論文)·· 9 天前AI 評分57
V-JEPA-LM:影片語言模型理解靜態文字對齊表徵以外的動態
V-JEPA-LM: Video-Language Models That Read Dynamics Beyond Static, Text-Aligned Representations
AI 導讀
研究團隊公開 Llama-V-JEPA-LM 開放影片語言模型,將凍結的 V-JEPA 2.1 世界模型編碼器與 Qwen3 / Llama(1B–8B)結合,並以反事實監督進行單階段訓練。
正文
We release Llama-V-JEPA-LM, open video-language models that join a frozen V-JEPA 2.1 world-model encoder to Qwen3 / Llama (1B–8B), trained in a single stage with counterfactual supervision: a clip and its transformed copy (reversed, mirrored, re-timed) get the same question with different answers, so only the change itself can explain the answer.
- Leads every size class on 11 video benchmarks (4B: 72.14 avg), strongest on dynamics (TOMATO, Vinoground, SSv2)
- Beats Meta's V-JEPA 2 / 2.1 video LLMs with 1/20 of the training data and 1/9 of the visual tokens
- 4.53M public samples, 32 visual tokens per frame, one training stage
All 6 checkpoints, code and a live demo are open.
來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co