跳到正文
HuggingFace Daily Papers(社區熱門論文)·· 2026-10-05AI 評分44

MEND: RL For Flow Models via Proximal Velocity Matching

MEND: RL For Flow Models via Proximal Velocity Matching

AI 導讀

MEND 提出一種以近端速度匹配為基礎的流模型強化學習方法,透過獎勵上限和位移成本篩選更新方向,不使用 KL 項、凍結參考模型或優勢權重。在 100 次更新後,MEND 在六項評估指標中的五項超越 Flow-GRPO,且與基礎模型圖像的距離相同;在相同預算下,它於四種訓練獎勵的每個評估更新點均勝過 ReFL 和 DiffusionNFT。

正文

Published on Oct 5

Authors:

,

,

,

Abstract

Reward post-training of flow models either reweights the model's own samples under a KL penalty or a frozen reference, often for thousands of updates, or backpropagates the reward and moves every sample without checking that the move is worth its size. We introduce MEND, a reinforcement learning method built on proximal velocity matching. MEND caps rewards within each prompt group, so samples that already score well receive no move. Below the cap, it proposes moves along the reward gradient and accepts one only when its capped reward gain exceeds a quadratic displacement price. The model then regresses onto the resulting velocity targets, with no KL term, frozen reference model, or advantage weights. In 100 updates, MEND outperforms Flow-GRPO (about 4k updates) on five of six evaluators at the same distance to base-model images. Under an equal-budget protocol, it surpasses ReFL and DiffusionNFT at every evaluated update across four training rewards, reaching PickScore 24.03 versus 23.92 and 23.43, respectively. A 300-update three-reward run also surpasses the five-reward DiffusionNFT model on all three rewards it trains on. MEND is general and easy to adopt: it applies to any flow backbone with a differentiable reward.

View arXiv page View PDF Project page GitHub Add to collection

來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co