MMEnterprise-Bench:評測企業智能體處理不斷演變的多模態紀錄
MMEnterprise-Bench: Evaluating Enterprise Agents over Evolving Multimodal Records
MMEnterprise-Bench 評測多模態智能體如何推理不斷演變的工作紀錄,並處理證據衝突與缺失。基準包含來自模擬協作項目的 1,613 條問題,分為證據推理、衝突解決及無法作答三類,並用於評估 10 個閉源及開源多模態 LLM。即使是表現最強的模型,遇到跨來源或時間衝突、或證據分散於多張圖片時仍有困難;增加檢索預算或使用更大型的檢索器,也未必能提高答案準確度。
Published on Oct 20
Authors:
,
,
,
,
,
Abstract
MMEnterprise-Bench evaluates multimodal agents on reasoning over evolving workplace records with conflicts and missing evidence.
Enterprise agents answer questions over workplace records that evolve through team collaboration. As information is shared, corrected, and revised across documents, chats, and figures, multiple versions of the same information accumulate in the workspace. Reliable question answering therefore requires agents to reason over sufficient evidence, resolve cross-source and temporal conflicts, and recognize when the available evidence is insufficient to establish an answer. To systematically evaluate these capabilities, we introduce MMEnterprise-Bench, a benchmark comprising 1,613 questions grounded in evolving multimodal records from simulated collaboration projects, covering three categories: Evidence Reasoning, Conflict Resolution, and Unanswerability. We evaluate 10 proprietary and open-source multimodal LLMs and find that even the strongest models struggle when evidence contains cross-source or temporal conflicts or spans multiple images. Moreover, increasing the retrieval budget or employing a larger retriever improves evidence recall but does not consistently translate into higher answer accuracy. These findings highlight the persistent challenges of reasoning over evolving workplace records and establish MMEnterprise-Bench as a testbed for developing more reliable enterprise agents. Code and data are available at https://mlvlab.github.io/MMEnterprise-Bench/.
View arXiv page View PDF Project page GitHub 1 Add to collection
Get this paper in your agent:
hf papers read 2610.27911
Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash
Models citing this paper 0
No model linking this paper
Cite arxiv.org/abs/2610.27911 in a model README.md to link it from this page.
Datasets citing this paper 0
No dataset linking this paper
Cite arxiv.org/abs/2610.27911 in a dataset README.md to link it from this page.
Spaces citing this paper 0
No Space linking this paper
Cite arxiv.org/abs/2610.27911 in a Space README.md to link it from this page.
Collections including this paper 0
No Collection including this paper
Add this paper to a collection to link it from this page.
來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co