跳到正文
HuggingFace Daily Papers(社區熱門論文)·· 10 天前AI 評分46

PixVid:將潛在影片擴散模型轉至像素空間

Back to Pixels: Lifting Latent Video Diffusion into Pixel Space

AI 導讀

研究提出 PixVid,將預訓練的潛在影片模型改為直接在像素空間生成每一幀。PixVid 以 Wan2.1-14B 微調,在約 1.3M 訓練片段下表現可媲美潛在空間基線。它可在每幀 token 數相同的情況下生成 2 倍分辨率影片,並在每次動作後於下一幀回應。

正文

Published on Nov 12

Authors:

,

,

,

,

,

,

,

,

,

,

Abstract

Frontier video generation models work in a VAE latent space that compresses video in both space and time. This compression made training tractable, but now limits high-resolution, controllable, and interactive generation -- doubling resolution quadruples token count, conditions apply to groups of frames rather than individual ones, and each response must wait for the rest of its multi-frame block. We present PixVid, a simple yet effective method that lifts a pretrained latent video model to directly generate every frame in pixel space. PixVid keeps the diffusion transformer and replaces the VAE decoder with a convolutional head that renders its outputs into pixels. We find that predicting clean video rather than velocity, together with a larger noise shift, is key to quality. Finetuned from Wan2.1-14B, PixVid is competitive with the latent baseline within sim 1.3M training clips. Working framewise in pixels, it generates at 2times higher resolution with the same per-frame token count, keeps depth-conditioned video editing aligned under fast motion, and responds to each action at the next frame. Project page: https://chenshuo20.github.io/PixVid/

View arXiv page View PDF Project page Add to collection

Models citing this paper 0

No model linking this paper

Cite arxiv.org/abs/2611.17336 in a model README.md to link it from this page.

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2611.17336 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2611.17336 in a Space README.md to link it from this page.

Collections including this paper 1

來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co