# Improving the Training of Rectified Flows

arXiv: https://arxiv.org/abs/2405.20320 · HTML: https://mlsanity.com/paper/2405.20320/

## Key facts

- Cited by 108 papers in the map (data through 14 Oct 2026).
- Placed in Flow matching, time series and science, one of 14 directions in Diffusion models.
- Builds on 51 of its 60 references that are in the map.
- First posted on arXiv 30 May 2024.

## Placed in

- Diffusion models / [Flow matching, time series and science](https://mlsanity.com/diffusion/flow-matching-time-series-and-science/)

## Builds on (most cited first, 30 of 51)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Flow Matching for Generative Modeling](https://mlsanity.com/paper/2210.02747/) (2022)
- [Scaling Rectified Flow Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2403.03206/) (2024)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow](https://mlsanity.com/paper/2209.03003/) (2022)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Elucidating the Design Space of Diffusion-Based Generative Models](https://mlsanity.com/paper/2206.00364/) (2022)
- [Large Scale GAN Training for High Fidelity Natural Image Synthesis](https://mlsanity.com/paper/1809.11096/) (2018)
- [Prompt-to-Prompt Image Editing with Cross Attention Control](https://mlsanity.com/paper/2208.01626/) (2022)
- [Video Diffusion Models](https://mlsanity.com/paper/2204.03458/) (2022)
- [DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps](https://mlsanity.com/paper/2206.00927/) (2022)
- [Consistency Models](https://mlsanity.com/paper/2303.01469/) (2023)
- [Training Generative Adversarial Networks with Limited Data](https://mlsanity.com/paper/2006.06676/) (2020)
- [Null-text Inversion for Editing Real Images using Guided Diffusion Models](https://mlsanity.com/paper/2211.09794/) (2022)
- [Mixed Precision Training](https://mlsanity.com/paper/1710.03740/) (2017)
- [StarGAN v2: Diverse Image Synthesis for Multiple Domains](https://mlsanity.com/paper/1912.01865/) (2019)
- [One-step Diffusion with Distribution Matching Distillation](https://mlsanity.com/paper/2311.18828/) (2023)
- [Building Normalizing Flows with Stochastic Interpolants](https://mlsanity.com/paper/2209.15571/) (2022)
- [DiffEdit: Diffusion-based semantic image editing with mask guidance](https://mlsanity.com/paper/2210.11427/) (2022)
- [DiffusionCLIP: Text-Guided Diffusion Models for Robust Image Manipulation](https://mlsanity.com/paper/2110.02711/) (2021)

## Built on by (most cited first, 30 of 108)

- [Taming Rectified Flow for Inversion and Editing](https://mlsanity.com/paper/2411.04746/) (2024)
- [Step-Video-T2V Technical Report: The Practice, Challenges, and Future of Video Foundation Model](https://mlsanity.com/paper/2502.10248/) (2025)
- [Rectified Diffusion: Straightness Is Not Your Need in Rectified Flow](https://mlsanity.com/paper/2410.07303/) (2024)
- [GestureLSM: Latent Shortcut based Co-Speech Gesture Generation with Spatial-Temporal Modeling](https://mlsanity.com/paper/2501.18898/) (2025)
- [Steering Rectified Flow Models in the Vector Field for Controlled Image Generation](https://mlsanity.com/paper/2412.00100/) (2024)
- [Harnessing Diffusion-Yielded Score Priors for Image Restoration](https://mlsanity.com/paper/2507.20590/) (2025)
- [Efficient Diffusion Models: A Survey](https://mlsanity.com/paper/2502.06805/) (2025)
- [Simple ReFlow: Improved Techniques for Fast Flow Models](https://mlsanity.com/paper/2410.07815/) (2024)
- [Diff2Flow: Training Flow Matching Models via Diffusion Model Alignment](https://mlsanity.com/paper/2506.02221/) (2025)
- [One-Step Flow Policy Mirror Descent](https://mlsanity.com/paper/2507.23675/) (2025)
- [Composition and Control with Distilled Energy Diffusion Models and Sequential Monte Carlo](https://mlsanity.com/paper/2502.12786/) (2025)
- [On the Relation between Rectified Flows and Optimal Transport](https://mlsanity.com/paper/2505.19712/) (2025)
- [Epsilon-VAE: Denoising as Visual Decoding](https://mlsanity.com/paper/2410.04081/) (2024)
- [Beyond Optimal Transport: Model-Aligned Coupling for Flow Matching](https://mlsanity.com/paper/2505.23346/) (2025)
- [Fast LiDAR Data Generation with Rectified Flows](https://mlsanity.com/paper/2412.02241/) (2024)
- [Turbo3D: Ultra-fast Text-to-3D Generation](https://mlsanity.com/paper/2412.04470/) (2024)
- [Flow Matching Meets Biology and Life Science: A Survey](https://mlsanity.com/paper/2507.17731/) (2025)
- [On the Convergence and Straightness of Rectified Flow](https://mlsanity.com/paper/2410.14949/) (2024)
- [Constant Acceleration Flow](https://mlsanity.com/paper/2411.00322/) (2024)
- [Analyzing and Mitigating Model Collapse in Rectified Flow Models](https://mlsanity.com/paper/2412.08175/) (2024)
- [Adams Bashforth Moulton Solver for Inversion and Editing in Rectified Flow](https://mlsanity.com/paper/2503.16522/) (2025)
- [CurveFlow: Curvature-Guided Flow Matching for Image Generation](https://mlsanity.com/paper/2508.15093/) (2025)
- [Autoregressive Distillation of Diffusion Transformers](https://mlsanity.com/paper/2504.11295/) (2025)
- [One-Step Offline Distillation of Diffusion-based Models via Koopman Modeling](https://mlsanity.com/paper/2505.13358/) (2025)
- [Improving Rectified Flow with Boundary Conditions](https://mlsanity.com/paper/2506.15864/) (2025)
- [CardiacFlow: 3D+t Four-Chamber Cardiac Shape Completion and Generation via Flow Matching](https://mlsanity.com/paper/2509.05754/) (2025)
- [Register Any Point: Scaling 3D Point Cloud Registration by Flow Matching](https://mlsanity.com/paper/2512.01850/) (2025)
- [Modular MeanFlow: Towards Stable and Scalable One-Step Generative Modeling](https://mlsanity.com/paper/2508.17426/) (2025)
- [Mirror Flow Matching with Heavy-Tailed Priors for Generative Modeling on Convex Domains](https://mlsanity.com/paper/2510.08929/) (2025)
- [Learning Straight Flows: Variational Flow Matching for Efficient Generation](https://mlsanity.com/paper/2511.17583/) (2025)

## Abstract (arXiv)

Diffusion models have shown great promise for image and video generation, but sampling from state-of-the-art models requires expensive numerical integration of a generative ODE. One approach for tackling this problem is rectified flows, which iteratively learn smooth ODE paths that are less susceptible to truncation error. However, rectified flows still require a relatively large number of function evaluations (NFEs). In this work, we propose improved techniques for training rectified flows, allowing them to compete with knowledge distillation methods even in the low NFE setting. Our main insight is that under realistic settings, a single iteration of the Reflow algorithm for training rectified flows is sufficient to learn nearly straight trajectories; hence, the current practice of using multiple Reflow iterations is unnecessary. We thus propose techniques to improve one-round training of rectified flows, including a U-shaped timestep distribution and LPIPS-Huber premetric. With these techniques, we improve the FID of the previous 2-rectified flow by up to 75% in the 1 NFE setting on CIFAR-10. On ImageNet 64×64, our improved rectified flow outperforms the state-of-the-art distillation methods such as consistency distillation and progressive distillation in both one-step and two-step settings and rivals the performance of improved consistency training (iCT) in FID. Code is available at https://github.com/sangyun884/rfpp.