跳到正文
HuggingFace Daily Papers(社區熱門論文)·· 2026-10-13AI 評分46

VideoJev: System One Model for Video Understanding and Interaction

VideoJev: System One Model for Video Understanding and Interaction

AI 導讀

VideoJev 將影片理解與互動建模為結構化決策任務,並以共享評分頭處理時間定位及選擇題式影片問答。在三個時間定位基準上,VideoJev-4B 和 VideoJev-35B-A3B 的平均 mIoU 分別較其骨幹模型高 6.7 和 10.8 分,推理速度提升 2.8-3.1 倍。在四個影片問答基準上,兩者於 Thinking mode 的表現與骨幹模型相若,推理速度提升逾 100 倍。

正文

Published on Oct 13

Authors:

,

,

,

,

,

,

,

,

,

,

,

,

,

,

,

,

Abstract

Visual agents must turn video observations into timely decisions to track progress and choose what to do next. Building on the emerging System One paradigm, we introduce VideoJev, a family of models that cast video understanding and interaction as structured decision tasks. We adapt pretrained multimodal large language models (MLLMs) by replacing autoregressive answer decoding with a shared head that scores temporal intervals and textual options against video evidence. These scores yield event spans or answer choices, unifying temporal grounding and multiple-choice video question answering. This combines the semantic flexibility of MLLMs with efficient decisions that agents can use directly. As an interaction unfolds, new goals and intermediate results prompt further queries about the same visual history. VideoJev therefore reuses visual computation through Cached inference for queries as they arrive and Joint inference for multiple queries in a single forward pass. Across three temporal grounding benchmarks, VideoJev-4B and VideoJev-35B-A3B improve mean mIoU over their backbones by 6.7 and 10.8 points, respectively, with measured inference speedups of 2.8-3.1times. On four video QA benchmarks, both achieve performance comparable to their backbones in Thinking mode, with measured inference speedups exceeding 100times. These results offer a unified, reusable decision interface for composing structured visual judgments into complex, open-ended workflows.

View arXiv page View PDF Project page GitHub 4 Add to collection

Get this paper in your agent:

hf papers read 2610.18615

Don't have the latest CLI?

curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 2

ZhuYuhan/VideoJev-4B

Feature Extraction •

4B •

Updated about 1 hour ago

•

25

•

4

ZhuYuhan/VideoJev-35B-A3B

Feature Extraction •

34B •

Updated about 1 hour ago

•

27

•

3

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2610.18615 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2610.18615 in a Space README.md to link it from this page.

Collections including this paper 1

來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co