跳到正文
HuggingFace Daily Papers(社區熱門論文)·· 9 天前AI 評分57

V-JEPA-LM:影片語言模型理解靜態文字對齊表徵以外的動態

V-JEPA-LM: Video-Language Models That Read Dynamics Beyond Static, Text-Aligned Representations

AI 導讀

研究團隊公開 Llama-V-JEPA-LM 開放影片語言模型,將凍結的 V-JEPA 2.1 世界模型編碼器與 Qwen3 / Llama(1B–8B)結合,並以反事實監督進行單階段訓練。

正文

We release Llama-V-JEPA-LM, open video-language models that join a frozen V-JEPA 2.1 world-model encoder to Qwen3 / Llama (1B–8B), trained in a single stage with counterfactual supervision: a clip and its transformed copy (reversed, mirrored, re-timed) get the same question with different answers, so only the change itself can explain the answer.

  • Leads every size class on 11 video benchmarks (4B: 72.14 avg), strongest on dynamics (TOMATO, Vinoground, SSv2)
  • Beats Meta's V-JEPA 2 / 2.1 video LLMs with 1/20 of the training data and 1/9 of the visual tokens
  • 4.53M public samples, 32 visual tokens per frame, one training stage

All 6 checkpoints, code and a live demo are open.

來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co