跳到正文
HuggingFace Daily Papers(社區熱門論文)·· 6 天前AI 評分45

BLIP-Live:平衡即時性與觸發精準度的主動式串流影片智能體

BLIP-Live: Balancing Timeliness & Firing Precision in Proactive Streaming Video Agents

AI 導讀

BLIP-Live 提出一套整合數據、評測指標、推理及訓練的框架,用於主動式串流影片理解,並建立包含人工標註的 BLIP-Live-780K 數據集及 PAUC-F 評測指標。BLIP-Live-Qwen 在四項主動式串流基準測試中達到最先進表現,較基線最高提升 44%,並勝過多個閉源 API 模型。程式碼及數據集將公開。

正文

Published on Nov 16

Authors:

,

,

,

,

,

,

Abstract

BLIP-Live unifies data, metrics, inference, and training for proactive streaming video understanding, using the PAUC-F metric and achieving state-of-the-art results on proactive streaming benchmarks.

Streaming video understanding is becoming increasingly important for real-world interactive systems. Such systems must be proactive for natural and useful human interaction: they should respond as soon as sufficient evidence becomes available while avoiding premature, redundant, or incorrect responses. We present BLIP-Live, a framework unifying data, metrics, harness, and training around a shared formulation of proactivity as effective causal response decisions balancing the two competing objectives of responding vs silence. First, we construct BLIP-Live-780K dataset containing human annotations of proactive response content and timing, which defines this desired behavior. We then propose PAUC-F, a metric to effectively measure our desired proactive behavior. PAUC-F combines existing response timeliness metrics with firing precision to reward early useful responses without rewarding excessive firing. With our desired behavior defined and measurable, we finally investigate model-agnostic inference harnesses that draw out proactive streaming capabilities of offline vision language models (VLMs), together with training pipelines that learn to further improve their proactive responsiveness. Across both settings, we find that combinations of sliding visual windows, online interaction memory, and structured response outputs consistently yield strongest proactive performance. Our experiments result in two models, with our BLIP-Live-Qwen model achieving state-of-the-art performance across four proactive streaming benchmarks, with gains up to 44% over baselines and outperforming multiple closed API models. Our code and data will be released publicly.

View arXiv page View PDF Project page Add to collection

Get this paper in your agent:

hf papers read 2611.21553

Don't have the latest CLI?

curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 0

No model linking this paper

Cite arxiv.org/abs/2611.21553 in a model README.md to link it from this page.

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2611.21553 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2611.21553 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.

來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co