跳到正文
HuggingFace Daily Papers(社區熱門論文)·· 2026-10-08AI 評分46

DiffusionART: Efficient, Stable and Scalable Reinforcement of Diffusion Models

DiffusionART: Efficient, Stable and Scalable Reinforcement of Diffusion Models

AI 導讀

DiffusionART 提出一種擴散模型線上強化學習方法,透過聚合每個提示詞的獎勵改善方向,實現高效穩定的訓練。在 HunyuanVideo-13B 上,DiffusionART 達到 DiffusionNFT 和 Flow-GRPO-Fast 後期平均獎勵水平所需的 GPU 小時分別少 12.02 倍和 11.72 倍,平均訓練獎勵則分別高 28.8% 和 53.0%。

正文

Published on Oct 8

Authors:

,

,

,

,

,

,

,

Abstract

Online reinforcement learning (RL) fine-tunes diffusion models with black-box rewards, but reverse-process methods such as Flow-GRPO are limited in efficiency by poor credit assignment, while forward-process methods such as DiffusionNFT are efficient yet unstable without carefully tuned exponential-moving-average (EMA) off-policy training. We show that NFT-style objectives contain a hidden forward--reverse consistency force that joint positive--negative training cancels only partially, and that EMA stabilizes training through restoration rather than off-policy rollouts. We introduce DiffusionART (Diffusion Reinforcement with Aggregation-First Reward Tilting), which aggregates each prompt's rewards into a shared reward-improving direction for reverse-process states, removing the consistency force by construction, and adds a trust-region-masked second update for drift-adaptive restoration without EMA tuning. It requires no differentiable reward model, remains effective with fewer samples per prompt, and supports pairwise reward models (DiffusionART-Pairwise). Across image and video generation, DiffusionART trains efficiently and stably without per-task tuning. In particular, on HunyuanVideo-13B with equally weighted visual and motion quality rewards, DiffusionART reaches the late-stage mean reward levels of DiffusionNFT and Flow-GRPO-Fast with 12.02 and 11.72 times fewer GPU hours, respectively, while achieving 28.8% and 53.0% higher mean training rewards.

View arXiv page View PDF Project page GitHub Add to collection

Get this paper in your agent:

hf papers read 2610.12572

Don't have the latest CLI?

curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 2

vanzl/DiffusionART-SD35M-MultiReward

Text-to-Image •

Updated about 2 hours ago

vanzl/DiffusionART-HunyuanVideo-7-2-1

Text-to-Video •

13B •

Updated 8 minutes ago

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2610.12572 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2610.12572 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.

來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co