跳到正文
HuggingFace Daily Papers(社區熱門論文)·· 2026-10-08AI 評分54

WOVEN 將視覺世界建模融入多模態大語言模型

WOVEN: Weaving Visual World Modeling into Multimodal LLMs

AI 導讀

研究團隊提出 WOVEN 視覺轉移推理基準及訓練資料,包含 36,076 個樣本,涵蓋 20 種場景、5 類動作和 8 類推理。研究團隊評估 38 個前沿多模態大語言模型,發現即使最強模型仍遠低於人類,且缺口跨模型系列、不同規模持續存在。

正文

Published on Oct 8

Authors:

,

,

,

,

,

,

,

,

,

,

Abstract

Multimodal large language models (MLLMs) struggle with spatial, embodied, physical, and temporal reasoning. We hypothesize that these failures reflect a shared deficit in visual transition reasoning, and test whether this capability can serve as a shared training primitive, one that different models can learn from different supervision sources and reuse across different tasks, with a systematic training recipe. Existing benchmarks document these deficits separately but do not support controlled comparisons across scenes, actions, and reasoning operations. We therefore introduce WOVEN, a training source and benchmark for visual transition reasoning that organizes transition supervision by scene, action, and reasoning type, using diverse, realistic rollouts from video-pretrained generative models: 36,076 examples across 20 scene types, 5 action types, and 8 reasoning types. We first evaluate 38 frontier MLLMs (e.g., GPT-5.4 and Qwen3-VL-235B-A22B) and find a substantial and systematic deficit: even the strongest models fall far below humans, and the failures recur across model families and persist with scale. We then train MLLMs at multiple scales on WOVEN and find that they learn a shared capability that transfers broadly: training subsets of only about 2,000 items each collectively improve 22 of 26 external benchmarks by up to 27.3 percentage points, and WOVEN data can replace 30-50% of a task's own training data with comparable accuracy. Controlled comparisons further yield a training recipe for visual world modeling, validated prospectively on held-out benchmarks: select supervision by the reasoning operation it teaches rather than by the actions, scenes, or domains it shows, and prefer larger changes to the visual state for robustness. Our work establishes visual transition reasoning as a reusable foundation for systematic visual world-model training in MLLMs.

View arXiv page View PDF Project page GitHub Add to collection

Get this paper in your agent:

hf papers read 2610.12417

Don't have the latest CLI?

curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 0

No model linking this paper

Cite arxiv.org/abs/2610.12417 in a model README.md to link it from this page.

Datasets citing this paper 1

MLL-Lab/WOVEN

Updated 4 days ago

•

61

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2610.12417 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.

來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co