Matrix-Game 4.0 利用影片模型實現即時可玩遊戲世界
Matrix-Game 4.0: Harnessing Video Models for Real-Time Playable Game Worlds
Matrix-Game 4.0 透過可執行遊戲引擎與自回歸影片渲染器分離遊戲邏輯和視覺生成,實現低延遲的即時互動遊戲。在單張 NVIDIA RTX5090 上,系統可提供 480p、60 FPS 遊戲體驗,按鍵至畫面顯示延遲為 112ms。單步生成的穩態吞吐量為 83FPS,峯值 GPU 記憶體用量為 18.2GiB;論文亦公開程式碼、模型權重及低多邊形遊戲素材。
Published on Oct 20
Authors:
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
Abstract
Matrix-Game 4.0 separates game logic from visual synthesis via an executable engine and autoregressive video renderer, achieving real-time interactive gameplay with low latency.
Moving from controllable videos to genuinely playable worlds requires persistent world state, executable game rules, and low-latency interaction. We outline a roadmap for the next generation of Matrix-Game that separates game execution from visual generation through an executable game harness, and introduce Matrix-Game 4.0 as the first system built on this design. A lightweight game engine maintains world state and executes interaction rules, a multimodal coding agent translates player ideas into game code and dynamic scene captions, and a renderer based on autoregressive video model transforms engine-rendered low-poly proxies into detailed visuals, conditioned on dynamic captions and memory references. This framework provides continuous guidance for visual generation and enables natural-language creation and modification of scenes and gameplay. Our renderer is trained on aligned low-poly and high-fidelity video pairs and distilled into a causal generator that generates in four-frame chunks with one or two denoising steps. Together with targeted inference optimizations, Matrix-Game 4.0 enables 480p, 60 FPS gameplay on a single NVIDIA RTX5090 with a key-press-to-display latency of 112ms. Single-step generation achieves 83FPS steady-state throughput with 18.2GiB of peak GPU memory usage. With fixed prompts, steady-state throughput also reaches 67FPS on an RTX5080 and 58FPS on an RTX5070Ti. We demonstrate customizable shooter and driving games with engine-enforced mechanics, and release code, model weights, and low-poly game assets to support research and creation of playable game worlds.
View arXiv page View PDF Project page Add to collection
Get this paper in your agent:
hf papers read 2610.27999
Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash
Models citing this paper 0
No model linking this paper
Cite arxiv.org/abs/2610.27999 in a model README.md to link it from this page.
Datasets citing this paper 0
No dataset linking this paper
Cite arxiv.org/abs/2610.27999 in a dataset README.md to link it from this page.
Spaces citing this paper 0
No Space linking this paper
Cite arxiv.org/abs/2610.27999 in a Space README.md to link it from this page.
Collections including this paper 0
No Collection including this paper
Add this paper to a collection to link it from this page.
來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co