跳到正文
HuggingFace Daily Papers(社區熱門論文)·· 2026-10-06AI 評分40

Consistent Distribution Matching for Data-Free Diffusion Distillation

Consistent Distribution Matching for Data-Free Diffusion Distillation

AI 導讀

CDMD 提出無需訓練輔助 fake-score 網絡的無數據擴散蒸餾方法,並將樣本生成與速度估計統一於單一學生網絡。該方法只需一個凍結教師、一個可訓練學生及一個目標函數,在 ImageNet 256×256 上以 1 NFE 達到 2.04 FID,優於 DMD2 的 3.43 FID。以 4 NFEs 推理時,CDMD 達到 1.37 FID,接近教師的 1.35 FID。

正文

Excited to share our work on Consistent Distribution Matching! 🚀

Inspired by Distribution Matching Distillation (DMD) and Score Identity Distillation (SiD), we ask a simple question: Can we achieve distribution matching without training an auxiliary fake-score network?

Our answer is yes!

CDMD takes a step further by unifying sample generation and velocity estimation within a single student network. This gives us a much simpler distillation framework:

One frozen teacher + one trainable student + one objective.

No external training data, teacher trajectory simulation, or auxiliary score networks in our data-free setting.

Stronger generative performance: 2.04 FID at 1 NFE on ImageNet 256×256, outperforming evaluated data-free baselines (DMD2: 3.43 FID).

Near-teacher quality: 1.37 FID at 4 NFEs, approaching the teacher's 1.35 FID.

We also provide theoretical insights connecting our consistency objective to distribution matching through Wasserstein convergence.

Our takeaway: Effective distribution matching doesn't have to come with complicated multi-network optimization.

We'd love to hear your thoughts and feedback!

來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co