FastFrames 提出互動級延遲的自回歸影片擴散模型
FastFrames: Video Generation at Interactive Latency
FastFrames 論文提出一系列快速自回歸影片擴散模型,目標是在不犧牲生成品質下達到互動級延遲。1.3B 版本在單張 B200 GPU 上每影格耗時 36 ms、達 28 RPS,生成品質媲美慢近 8x 的 SOTA 模型;14B 版本在 8x B200 GPU 節點上每影格耗時 45 ms、達 22 RPS,並呈現顯著更佳的視覺品質和真實感。
Published on Nov 17
Authors:
,
,
,
,
,
Abstract
Autoregressive video diffusion models generate frames based on new observations and inputs, making them a promising medium for interactive visual experiences. However, existing models are orders of magnitude too slow for fluid interaction. This paper introduces FastFrames, a family of fast autoregressive video diffusion models that achieves interactive latency without compromising quality. We profile the inference pipeline of state-of-the-art models and find that operations often treated as fixed implementation details, such as VAE decoding and KV cache updates, in fact account for over half of end-to-end inference latency. We challenge the assumption that these components are indispensable and replace them with lightweight alternatives. Our 1.3B variant runs at 36 ms per frame, or 28 responses per second (RPS), on a single B200 GPU while matching the generation quality of a state-of-the-art model that is nearly 8x slower. Our 14B variant runs at 45 ms per frame, or 22 RPS, on an 8x B200 GPU node and yields significantly better visual quality and realism. We show that FastFrames enables real-time interaction in applications such as camera-conditioned and physics-conditioned video generation.
View arXiv page View PDF Project page GitHub Add to collection
Get this paper in your agent:
hf papers read 2611.23226
Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash
Models citing this paper 2
Text-to-Video • Updated 41 minutes ago • 91apple/FastFrames-L
Text-to-Video • Updated 41 minutes ago • 18apple/FastFrames-S
Datasets citing this paper 0
No dataset linking this paper
Cite arxiv.org/abs/2611.23226 in a dataset README.md to link it from this page.
Spaces citing this paper 1
Collections including this paper 0
No Collection including this paper
Add this paper to a collection to link it from this page.
來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co