On Variance Reduction in Learning Mean Flows
On the map: Diffusion models → Flow matching, time series and science
Abstract (arXiv)
One-step generative modeling has emerged as a leading approach for amortizing the inference cost of diffusion and flow-matching models. Among distillation-free methods, MeanFlow training is notoriously unstable, with non-decreasing loss and unbounded gradient variance. In this work, we establish a theory that attributes this pathology to a misuse of the conditional velocity field. We show that the conditional velocity plays two distinct statistical roles in the loss: both as an unbiased regression target and as a Monte Carlo control variate in a Jacobi-vector product, with the original MeanFlow loss assigning the wrong coefficient to the latter. We derive the optimal coefficient in closed form and show that a family of fixes in concurrent works corresponds to different practical realizations of the same optimum. A controlled sweep of this coefficient on two-dimensional benchmarks and on a latent Diffusion Transformer recovers the predicted bias-variance ordering. Our DiT experiment also reveals a quantitative FID-MSE landscape mismatch. Specifically, although the gradient-MSE is minimized at an interior coefficient value near β!=!0.94, the coefficient that minimizes FID prefers to use conditional velocity directly at the unbiased corner. Our analysis therefore explains why MeanFlow is unstable and unifies its concurrent remedies, and shows that the variance-optimal coefficient need not coincide with the quality-optimal one.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Flow matching, time series and science, one of 14 directions in Diffusion models. Builds on 28 of its 39 references that are in the map. First posted on arXiv 10 May 2026.
Builds on
28 of 39 references.
Closest ancestors · 6
Same direction, newest first
- 2026Stable Velocity: A Variance Perspective on Flow Matching5 Feb
- 2026Temporal Pair Consistency for Variance-Reduced Flow Matching4 Feb
- 2025On the Closed-Form of Flow Matching: Generalization Does Not Arise from Target Stochasticity4 Jun
- 2024Flow map matching with stochastic interpolants: A mathematical framework for consistency models11 Jun
- 2024Improving the Training of Rectified Flows30 May
- 2023Improving and generalizing flow-based generative models with minibatch optimal transport1 Feb
Other references · 22
- 2020Denoising Diffusion Probabilistic Modelscited by 21,076
- 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
- 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
- 2022Scalable Diffusion Models with Transformerscited by 6,054
- 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
- 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
- 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,101
- 2018Neural Ordinary Differential Equationscited by 3,779
- 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 3,045
- 2015Variational Inference with Normalizing Flowscited by 2,170
- 2016Density estimation using Real NVPcited by 2,042
- 2023Consistency Modelscited by 1,822
10 more
- 2023One-step Diffusion with Distribution Matching Distillationcited by 1,019
- 2022Building Normalizing Flows with Stochastic Interpolantscited by 995
- 2023Stochastic Interpolants: A Unifying Framework for Flows and Diffusionscited by 939
- 2024Improved Distribution Matching Distillation for Fast Image Synthesiscited by 684
- 2018FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Modelscited by 594
- 2025Mean Flows for One-step Generative Modelingcited by 561
- 2024One Step Diffusion via Shortcut Modelscited by 391
- 2025Improved Mean Flows: On the Challenges of Fastforward Generative Modelscited by 110
- 2025Decoupled MeanFlow: Turning Flow Models into Flow Maps for Accelerated Samplingcited by 20
- 2025Modular MeanFlow: Towards Stable and Scalable One-Step Generative Modelingcited by 8
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026A Unified Measure-Theoretic View of Diffusion, Score-Based, and Flow Matching Generative Models
- 2026Mean Velocity Matching: Rethinking Generative Dynamics in Diffusion Models
- 2025Generative Modeling with Continuous Flows: Sample Complexity of Flow Matching
- 2025Gradient Variance Reveals Failure Modes in Flow-Based Generative Models
- 2025Diff2Flow: Training Flow Matching Models via Diffusion Model Alignment
- 2025SoFlow: Solution Flow Models for One-Step Generative Modeling
- 2025Theoretical Guarantees for High Order Trajectory Refinement in Generative Flows
- 2026Stabilizing, Scaling & Enhancing MeanFlow for Large-scale Diffusion Distillation