VideoMaMa++:以 Preserve-and-Refine Token 實現時間一致的影片摳圖
VideoMaMa++: Temporally Consistent Video Matting via Preserve-and-Refine Token
VideoMaMa++提出以 Preserve-and-Refine Token 為核心的擴散式影片摳圖框架,改善長影片跨區塊的時間一致性及細節準確度。框架以分離的 RGB 與遮罩 token 流、可學習嵌入和滑動窗口傳遞遮罩,並在各區塊使用共用參考錨點,無需額外訓練;在標準基準及真實長影片上優於既有擴散式方法。
Published on Oct 19
Authors:
,
,
,
Abstract
A diffusion-based video matting framework uses separate token streams and learnable embeddings to propagate mattes across chunks, improving temporal consistency and detail accuracy on long videos.
Diffusion-based video matting has recently emerged as a leading approach, leveraging video diffusion priors from internet-scale pretraining to achieve strong zero-shot generalization from limited synthetic data. However, the heavy compute of these diffusion backbones limits inference to a short chunk of just 6 to 24 frames, so even ordinary videos span multiple chunks. Because each chunk is matted independently from coarse mask guidance, existing methods fail to maintain consistency across chunk boundaries, producing flickering edges and discontinuous transparency. We address this with a mask-guided diffusion-matting framework built on two ideas. The RGB and mask conditioning are represented as separate token streams that interact through full spatio-temporal self-attention, letting the model adaptively combine fine RGB texture with coarse mask layout rather than committing to a fixed channel-wise fusion. On top of this, we propose a novel pair of learnable embeddings, the preserve token and the refine token, that act as a per-frame conditioning interface and enable a sliding-window inference scheme in which already-generated mattes are propagated across chunk boundaries for local temporal consistency, complemented by a globally shared reference anchor at every chunk to prevent error accumulation in long sequences without any additional training. Across standard video matting benchmarks and in-the-wild long footage, our framework outperforms prior diffusion-based methods in both fine-detail accuracy and temporal consistency, and uniquely maintains consistency across chunk boundaries on long videos.
View arXiv page View PDF Project page GitHub Add to collection
Get this paper in your agent:
hf papers read 2610.26857
Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash
Models citing this paper 0
No model linking this paper
Cite arxiv.org/abs/2610.26857 in a model README.md to link it from this page.
Datasets citing this paper 0
No dataset linking this paper
Cite arxiv.org/abs/2610.26857 in a dataset README.md to link it from this page.
Spaces citing this paper 0
No Space linking this paper
Cite arxiv.org/abs/2610.26857 in a Space README.md to link it from this page.
Collections including this paper 0
No Collection including this paper
Add this paper to a collection to link it from this page.
來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co