CoWorld Reasoner:運用持續演進的影片世界模型驅動框架進行智能體多模態推理
CoWorld Reasoner: Agentic Multimodal Reasoning with the Evolving Video World Model-Driven Harness
CoWorld Reasoner 提出以影片世界模型作為視覺推理引擎的智能體框架,在七項基準測試中勝過同一 VLM 的直接推理。最強版本在 MMSI 和 ERQA 的相對準確率分別提升 11.8% 和 13.0%,並在五項基準測試中勝過使用相同骨幹模型的五個演進基線。
Published on Oct 15
Authors:
,
,
,
,
Abstract
CoWorld Reasoner is an agentic framework that uses a video world model as a flexible visual reasoning engine, enabling vision-language models to perform diverse spatiotemporal reasoning through guided visual imagination and co-evolving tools.
As Vision-Language Models (VLMs) advance, they are increasingly expected to perform spatiotemporal reasoning over motion, actions, and view changes beyond what the input images show. Video World Models (VWMs) provide a natural unified engine for visual imagination by generating task-relevant visual content as additional evidence. However, existing approaches typically invoke a VWM through fixed pipelines centered on a predefined form of visual imagination, such as novel-view synthesis. To support flexible visual imagination for diverse reasoning tasks, we introduce CoWorld Reasoner, an agentic framework that uses a VWM as a unified visual reasoning engine. Its Video World Model-Driven Harness exposes multiple visual operations through tools and uses skills to guide the VLM in deciding whether to imagine, selecting an operation, constructing a generation request, and interpreting the resulting visual evidence. We further introduce VLM-VWM Harness Co-Evolution, which jointly revises tools, skills, and execution code based on execution feedback while keeping both model weights fixed. Across seven benchmarks spanning spatial intelligence, temporal and causal reasoning, and perceptual reasoning, CoWorld Reasoner outperforms direct reasoning with the same VLM. The strongest variant improves on all seven benchmarks, including relative accuracy gains of 11.8% on MMSI and 13.0% on ERQA, and outperforms all five evolving baselines using the same backbone across five benchmarks.
View arXiv page View PDF Project page GitHub Add to collection
Models citing this paper 0
No model linking this paper
Cite arxiv.org/abs/2610.21910 in a model README.md to link it from this page.
Datasets citing this paper 1
ModalityDance/CoWorld-Reasoner-Dev
Viewer • Updated about 2 hours ago • 120 • 4
Spaces citing this paper 0
No Space linking this paper
Cite arxiv.org/abs/2610.21910 in a Space README.md to link it from this page.
Collections including this paper 0
No Collection including this paper
Add this paper to a collection to link it from this page.
來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co