WorldCast: Exploring Intelligent Visual Generation in World State Prediction
WorldCast: Exploring Intelligent Visual Generation in World State Prediction
WorldCast研究提出多模態生成模型的四級能力層級,並設立評估多步世界狀態預測的基準測試。基準涵蓋八個類別的320個人工設計案例,評估12個具代表性的模型;GPT-Image-2.5 System 得分72.8,Nano Banana Pro 得分42.8,最佳開源模型得分20.1。結果顯示,開源模型在複雜脈絡、交織參照及推理密集任務上仍有明顯差距。
Published on Oct 14
·
on Oct 16
Authors:
,
,
,
,
,
,
,
,
,
,
Abstract
Researchers propose a four-level hierarchy for multimodal generative models and introduce the WorldCast benchmark to evaluate multi-step world state prediction, revealing rapid progress by frontier systems but substantial gaps in open-source models for reasoning-heavy visual forecasting.
Humans interpret multimodal observations of the world and forecast how it will evolve. Recent multimodal generative systems have demonstrated similar capabilities: they can integrate multimodal context and synthesize visual outputs that depict changes in the world. To characterize this emerging trajectory, we organize model capabilities into a four-level hierarchy and discover that higher-level tasks remain relatively underexplored despite model progress. To bridge this gap, we explore model performance on multi-step world state prediction, a high-level task that requires models to forecast successive future world states from multimodal observations and diverse references. Such a task requires models to possess entangled understanding and generation capabilities, as well as extensive world knowledge to translate human action sequences, physical transformations, and abstract visual reasoning into coherent visual outputs. We introduce WorldCast, a benchmark comprising 320 manually designed cases across eight categories, and evaluate 12 representative models. The conducted experiments yield three key insights: (a) advances in frontier models are rapidly pushing the boundaries of intelligent world modeling, with the GPT-Image-2.5 System scoring 72.8 and demonstrating early-stage visual state prediction capabilities on real-world tasks; (b) WorldCast reveals substantial gaps between models: even Nano Banana Pro reaches only 42.8 and the best open-source model scores 20.1, with deficits in handling complex context, entangled references, and reasoning-heavy tasks; (c) strong generative models effectively draw on the capabilities of their underlying language models. These findings highlight the need for multimodal generative models that are capable of dynamic planning and reasoning while also being deeply unified to faithfully simulate real-world states.
View arXiv page View PDF Project page GitHub Add to collection
Get this paper in your agent:
hf papers read 2610.20731
Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash
Models citing this paper 0
No model linking this paper
Cite arxiv.org/abs/2610.20731 in a model README.md to link it from this page.
Datasets citing this paper 1
wenhaoyan77/WorldCast
Viewer • Updated about 2 hours ago • 320 • 1
Spaces citing this paper 0
No Space linking this paper
Cite arxiv.org/abs/2610.20731 in a Space README.md to link it from this page.
Collections including this paper 0
No Collection including this paper
Add this paper to a collection to link it from this page.
來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co