正規化軌跡模型（NTM）
Normalizing Trajectory Models
Normalizing Trajectory Models（NTM）以具表達力的條件正規化流建模每個反向步驟，並以精確似然訓練。模型的精確軌跡似然支援自蒸餾，讓輕量去噪器在四步內生成高質素樣本。在文字生成圖像基準測試中，NTM 以四個採樣步驟達到或超越強勁基線，並且是唯一保留生成軌跡精確似然的模型。
AuthorsJiatao Gu†, Tianrong Chen, Ying Shen‡**, David Berthelot, Shuangfei Zhai, Josh Susskind
Diffusion-based models decompose sampling into many small Gaussian denoising steps, an assumption that breaks down when generation is compressed to a few coarse transitions. Existing few-step methods address this through distillation, consistency training, or adversarial objectives, but sacrifice the likelihood framework in the process. We introduce Normalizing Trajectory Models (NTM), which models each reverse step as an expressive conditional normalizing flow with exact likelihood training. Architecturally, NTM combines shallow invertible blocks within each step with a deep parallel predictor across the trajectory, forming an end-to-end network trainable from scratch or initializable from pretrained flow-matching models. Its exact trajectory likelihood further enables self-distillation: a lightweight denoiser trained on the score function induced by the model itself produces high-quality samples in four steps. On text-to-image benchmarks, NTM matches or outperforms strong image generation baselines in just four sampling steps while uniquely retaining exact likelihood over the generative trajectory.
- † University of Pennsylvania
- ‡ UIUC
- ** Work done while at Apple
Related readings and updates.
Discrete flow matching generates text by iteratively transforming noise tokens into coherent language, but may require hundreds of forward passes. Distillation uses the multi-step trajectory to train a student to reproduce the process in a few steps. When the student underperforms, the usual explanation is insufficient capacity. We argue the opposite: the trajectory is the bottleneck, not the student. Each training trajectory is built through a…
Normalizing Flows (NFs) are a classical family of likelihood-based methods that have received revived attention. Recent efforts such as TARFlow have shown that NFs are capable of achieving promising performance on image modeling tasks, making them viable alternatives to other methods such as diffusion models. In this work, we further advance the state of Normalizing Flow generative models by introducing iterative TARFlow (iTARFlow). Unlike…
來源：Apple Machine Learning Research · machinelearning.apple.com