MInfoVisQA:資訊密集型視覺問答的多語言基準
MInfoVisQA: A Multilingual Benchmark for Information-Rich Visual Question Answering
MInfoVisQA 是一個涵蓋 24 種語言的平行基準,評估多模態大語言模型在資訊密集型視覺問答中的跨語言穩健性。基準包含 8,960 組問答配置,研究評估了 25 個模型;兩個語言對齊問答平均準確率同為 80.4% 的模型,跨語言表現差距分別達 7.8 和 20.3 個百分點。固定使用英文提問時,25 個模型在非英語視覺素材上的得分均較低。
Published on Oct 14
Authors:
,
,
,
,
Abstract
MInfoVisQA, a 24-language parallel benchmark, reveals substantial cross-language robustness gaps in multimodal large language models for information-rich visual question answering.
Multimodal large language models (MLLMs) excel in vision-language tasks; yet, whether they maintain consistent understanding of information-rich visuals (InfoVis) when the surface language changes remains unclear. To quantify this capability, we introduce MInfoVisQA, a parallel benchmark with 8,960 QA configurations derived from 128 seed instances across 24 languages. We construct seed-matched, language-parallel visuals and QA pairs by disentangling numerical-structural data from text and programmatically synchronizing translated labels. We evaluate 25 MLLMs under two settings: Language-Aligned QA (LQA, matching visual-text languages) and Cross-Language QA (XQA, fixed QA language with varying visual languages). The results show that aggregate accuracy masks substantial robustness gaps: two models with comparable mean LQA accuracy (80.4\%) exhibit performance spreads of 7.8 and 20.3 percentage points across languages. Moreover, all 25 models score lower with non-English visuals under fixed English QA, underscoring the need for cross-language sensitivity metrics.
View arXiv page View PDF Project page GitHub 2 Add to collection
Get this paper in your agent:
hf papers read 2611.21644
Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash
Models citing this paper 0
No model linking this paper
Cite arxiv.org/abs/2611.21644 in a model README.md to link it from this page.
Datasets citing this paper 1
arnodjiang/MInfoVisQA
Viewer • Updated about 8 hours ago • 8.96k • 570 • 1
Spaces citing this paper 1
Collections including this paper 0
No Collection including this paper
Add this paper to a collection to link it from this page.
來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co