SpecKCast:以頻域 Kalman 預測加速 Diffusion 推理
SpecKCast: Spectral Kalman Forecasting for Diffusion Acceleration
SpecKCast 是一種免訓練方法,透過頻域預測 Diffusion Transformer 的輸出,加速推理。它以 2D 離散餘弦轉換處理輸出,並用逐頻帶 Kalman filter 預測;在 FLUX.1-dev、Qwen-Image、Wan2.1-14B 和 HunyuanVideo 上,速度最高提升 6x,額外記憶體用量低於 0.05 GiB。
Published on Oct 19
Authors:
,
,
,
Abstract
SpecKCast forecasts diffusion transformer outputs in the frequency domain using per-band Kalman filters and adaptive step scheduling to accelerate inference with minimal memory overhead.
Diffusion Transformers (DiTs) achieve high-fidelity image and video generation, but they suffer from substantial inference costs due to repeated transformer evaluations across many denoising steps. Feature caching reduces this cost by reusing features across denoising steps. Recent forecasting methods extend this idea by extrapolating features to skipped steps instead of reusing them. However, existing forecasting methods extrapolate the features they predict uniformly, even though the spatial-frequency components of these features change most at different stages of denoising. They also overlook that denoising progresses nonuniformly, so equal step intervals can correspond to very different amounts of change. These mismatches amplify forecast errors as more steps are skipped, limiting the achievable acceleration. We propose SpecKCast, a training-free method that forecasts the model output in the frequency domain. SpecKCast applies a 2D discrete cosine transform (DCT) to the output and tracks each coefficient with a lightweight Kalman filter along a band-specific time coordinate derived from the noise schedule. The filter tracks each coefficient's value and rate of change, and the transformer is evaluated only when the filter's forecast uncertainty exceeds a threshold. Experiments on FLUX.1-dev, Qwen-Image, Wan2.1-14B, and HunyuanVideo show that SpecKCast achieves a better trade-off between acceleration and fidelity than existing caching and forecasting methods, reaching up to 6x speedup with less than 0.05 GiB of additional memory.
View arXiv page View PDF Project page Add to collection
Get this paper in your agent:
hf papers read 2610.26837
Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash
Models citing this paper 0
No model linking this paper
Cite arxiv.org/abs/2610.26837 in a model README.md to link it from this page.
Datasets citing this paper 0
No dataset linking this paper
Cite arxiv.org/abs/2610.26837 in a dataset README.md to link it from this page.
Spaces citing this paper 0
No Space linking this paper
Cite arxiv.org/abs/2610.26837 in a Space README.md to link it from this page.
Collections including this paper 0
No Collection including this paper
Add this paper to a collection to link it from this page.
來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co