Persistent Planar Memory for Video World Models
Persistent Planar Memory for Video World Models
PPMem 是一種影片世界模型,以平面基元作為持續 3D 記憶,生成具明確度量幾何資訊的長篇連貫影片。它在 DL3DV 和 RealEstate10K 上的影片品質優於先前以 3D 為基礎的世界模型,並同時產生度量深度、表面法線及可匯出的平面幾何。模型可在視點大幅變化時維持數百影格的連貫性。
Published on Oct 16
Authors:
,
,
,
,
,
Abstract
PPMem is a video world model that uses compact plane primitives as persistent 3D memory to generate long coherent videos with explicit metric geometry.
Video world models that generate and explore environments at scale need a persistent 3D memory of what they have produced. Plane primitives offer a natural candidate: a compact set of oriented planes captures the dominant structural elements (e.g., walls, floors, facades) of indoor and outdoor scenes, while requiring only 7 parameters per primitive. We present PPMem, a video world model that adopts plane primitives as its persistent 3D memory. PPMem augments a video diffusion transformer with a planar prediction head and a Depth Token Fusion (DTF) module: each autoregressive chunk jointly produces RGB video and planar geometry, which merges into a persistent plane memory that conditions subsequent chunks via rendered depth. The resulting memory is three orders of magnitude more compact than per-pixel point clouds (~100K primitives, 5MB for a full scene), geometrically explicit (metric depth, normals, exportable surfaces), cross-view consistent (jointly predicted by the shared backbone), and adaptive (refined by the merging procedure as generation proceeds). On DL3DV and RealEstate10K, PPMem outperforms prior 3D-grounded world models in video quality while jointly producing metric-scale depth, surface normals, and exportable planar geometry, sustaining coherence across hundreds of frames under large viewpoint changes. Project page: https://hyzcluster.github.io/ppmem.
View arXiv page View PDF Project page GitHub Add to collection
Get this paper in your agent:
hf papers read 2610.23795
Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash
Models citing this paper 1
Image-to-Video • 6B • Updated 21 minutes agoRobbyant-Research/PPMem
Datasets citing this paper 0
No dataset linking this paper
Cite arxiv.org/abs/2610.23795 in a dataset README.md to link it from this page.
Spaces citing this paper 0
No Space linking this paper
Cite arxiv.org/abs/2610.23795 in a Space README.md to link it from this page.
Collections including this paper 0
No Collection including this paper
Add this paper to a collection to link it from this page.
來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co