跳到正文
HuggingFace Daily Papers(社區熱門論文)·· 5 天前AI 評分45

GameRealm:以互動遊戲為視覺學習提供基礎

GameRealm: Grounding Visual Learning in Interactive Games

AI 導讀

GameRealm 是大型多模態遊戲資料集,收錄超過 120 million 個 RGB 影格及 1,000 hours 遊戲片段,並以同步標註將視覺觀察與遊戲中的監督訊號連結。資料集涵蓋深度、相機參數、音訊、控制輸入及 3D 點軌跡,並支援單視角、立體、多視角及全景錄製。實驗顯示,GameRealm 能提升視覺基礎模型在視覺生成及幾何學習任務上的表現。

正文

Published on Nov 17

Authors:

,

,

,

,

,

,

,

,

,

,

Abstract

GameRealm is a large-scale multimodal gameplay dataset with synchronized depth, camera, audio, control, and 3D trajectory annotations that improves visual foundation models for generation and geometric learning.

Visual foundation models are central to physical intelligence, yet their development remains constrained by fragmented training datasets, inconsistent annotation quality, and limited multimodal supervision. To address these challenges, we introduce GameRealm, a large-scale, richly annotated dataset that grounds visual observations in synchronized, game-native supervision. GameRealm encompasses over 120 million RGB frames and 1,000 hours of gameplay footage across all views. Beyond its unprecedented scale, GameRealm provides rich multimodal supervision, including metric depth, camera parameters, synchronized audio, control inputs, and 3D point trajectories. Its diversity further extends to flexible view configurations, spanning single-view, stereo, multi-view, and panoramic recordings. To enable scalable acquisition across diverse game environments, we develop a unified data collection pipeline that combines human-operated interaction with agentic exploration. Extensive experiments across diverse visual learning tasks demonstrate that GameRealm consistently strengthens visual foundation models in both visual generation and geometric learning. These results establish GameRealm as a versatile and valuable data resource, unlocking new opportunities for advancing visual intelligence.

View arXiv page View PDF Project page GitHub 8 Add to collection

Models citing this paper 0

No model linking this paper

Cite arxiv.org/abs/2611.23140 in a model README.md to link it from this page.

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2611.23140 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2611.23140 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.

來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co