跳到正文
HuggingFace Daily Papers(社區熱門論文)·· 27 天前AI 評分47

Omni-InteractScope:評測連續全模態串流中的主動與智能體互動能力

Omni-InteractScope: Benchmarking Proactive and Agentic Interaction in Continuous Omnimodal Streams

AI 導讀

Omni-InteractScope 是一項評測 Omni-LLMs 在連續影音串流中互動可靠性的基準,涵蓋 1,479 個人工標註互動、1,790 個時間目標及八組任務。測試的 12 個模型中,工具調用的準時首答比例均不超過 40%,時間與工具聯合準確率最高為 32.0%;提醒召回率不超過 23.0%。9 個模型未能完整覆蓋任何一段指引序列。

正文

Published on Oct 26

Authors:

,

,

,

,

,

,

,

,

,

,

,

,

Abstract

Omni-InteractScope evaluates continuous audiovisual interaction reliability, revealing that current Omni-LLMs struggle with timely tool use, query-free reminders, and sustained guidance.

Omnimodal large language models (Omni-LLMs) are increasingly expected to assist users while observing continuous audiovisual streams. However, existing evaluations provide limited insight into how reliably these models connect stream evidence and user needs to timely, sustained interventions. We introduce Omni-InteractScope, a benchmark of 1,479 human-annotated, cross-reviewed interactions with 1,790 temporal targets across eight task groups. It links input modalities, event triggers, intervention types, and user needs, covering event-timed tool invocation and query-free assistance alongside notifications, warnings, state tracking, and continuous responses. Our experimental results on the proposed benchmark reveal three critical limitations of current Omni-LLMs: (1) Timely tool invocation remains unreliable: all 12 evaluated models produce an on-time first response for at most 40\% of tool-invocation targets, and joint timing-and-tool accuracy reaches at most 32.0\%. (2) Query-free reminders remain poorly covered: all evaluated models achieve lower timely reference-target recall on query-free reminders than on explicit questions, measured on their respective target sets. Reminder recall never exceeds 23.0\%. (3) End-to-end guidance coverage remains limited: nine models cover none of the 26 guidance sequences in full, and the remaining three cover only one to three sequences each. Targeted controls further show that tool-invocation scores alone do not establish event grounding and that reminder coverage depends on response history and frequency. These findings motivate developing and evaluating streaming assistants for timely action, initiative under general user needs, and coverage of entire response sequences.

View arXiv page View PDF Project page GitHub 3 Add to collection

Get this paper in your agent:

hf papers read 2610.35439

Don't have the latest CLI?

curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 0

No model linking this paper

Cite arxiv.org/abs/2610.35439 in a model README.md to link it from this page.

Datasets citing this paper 1

Darden7/Omni-InteractScope

Viewer •

Updated about 20 hours ago

•

1.48k

•

180

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2610.35439 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.

來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co