LightVAE: Towards Compact and Efficient Video Autoencoders
LightVAE: Towards Compact and Efficient Video Autoencoders
LightVAE 提出影片 VAE 壓縮框架,透過時序低秩壓縮、區塊剪枝及特徵對齊,加快影片解碼。對 Wan VAE 系列,其解碼吞吐量最高達 NVIDIA H100 和 RTX 5090 上高效 VAE 解碼器的 2.24 倍及 1.11 倍,且 DAVIS PSNR 較高、LPIPS 較低。
Published on Oct 10
Authors:
,
,
,
,
,
,
Abstract
Latent diffusion models enable high-quality video generation, but video VAE encoding and decoding remain computationally expensive. Existing methods use stage-specific lightweight operators, yet either constrain spatial and temporal computation jointly or remove temporal computation altogether, without distinguishing their compression sensitivities in pretrained Conv3D. To address these limitations, we present LightVAE, a compression framework that exploits redundancy in temporal modeling and network depth. LightVAE exactly reparameterizes eligible causal Conv3D layers into an aggregate spatial response and a difference-driven temporal correction. Based on their distinct compression sensitivities, temporal low-rank compression (TLR) keeps the aggregate spatial kernel dense and applies layer-specific low-rank approximation only to the temporal correction. An algebraically equivalent folded implementation converts the arithmetic savings into practical acceleration. LightVAE further uses width-preserving block pruning to reduce depth and probe-guided feature alignment, guided by feedback from a frozen teacher suffix, to recover quality after compression. The framework also extends to video encoders. Across three video VAE backbones and two GPU platforms, LightVAE consistently accelerates decoding. For the Wan VAE family, it achieves up to 2.24times and 1.11times the decoding throughput of state-of-the-art efficient VAE decoders on NVIDIA H100 and RTX~5090, respectively, with higher DAVIS PSNR and lower LPIPS. On MiniMax-H3, it achieves up to 2.97times the original decoding throughput while retaining 99.51\% of the original model's overall VBench-2.0 score. Our code is available at https://github.com/ModelTC/LightVAE.
View arXiv page View PDF GitHub Add to collection
Get this paper in your agent:
hf papers read 2610.14726
Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash
Models citing this paper 0
No model linking this paper
Cite arxiv.org/abs/2610.14726 in a model README.md to link it from this page.
Datasets citing this paper 0
No dataset linking this paper
Cite arxiv.org/abs/2610.14726 in a dataset README.md to link it from this page.
Spaces citing this paper 0
No Space linking this paper
Cite arxiv.org/abs/2610.14726 in a Space README.md to link it from this page.
Collections including this paper 0
No Collection including this paper
Add this paper to a collection to link it from this page.
來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co