# Preconditioned Flow Matching

arXiv: https://arxiv.org/abs/2603.02337 · HTML: https://mlsanity.com/paper/2603.02337/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Flow matching, time series and science, one of 14 directions in Diffusion models.
- Builds on 26 of its 39 references that are in the map.
- First posted on arXiv 2 Mar 2026.

## Placed in

- Diffusion models / [Flow matching, time series and science](https://mlsanity.com/diffusion/flow-matching-time-series-and-science/)

## Builds on (most cited first, 26 of 26)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Flow Matching for Generative Modeling](https://mlsanity.com/paper/2210.02747/) (2022)
- [Elucidating the Design Space of Diffusion-Based Generative Models](https://mlsanity.com/paper/2206.00364/) (2022)
- [Density estimation using Real NVP](https://mlsanity.com/paper/1605.08803/) (2016)
- [LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop](https://mlsanity.com/paper/1506.03365/) (2015)
- [Normalizing Flows for Probabilistic Modeling and Inference](https://mlsanity.com/paper/1912.02762/) (2019)
- [StarGAN v2: Diverse Image Synthesis for Multiple Domains](https://mlsanity.com/paper/1912.01865/) (2019)
- [A Downsampled Variant of ImageNet as an Alternative to the CIFAR datasets](https://mlsanity.com/paper/1707.08819/) (2017)
- [Flow Matching Guide and Code](https://mlsanity.com/paper/2412.06264/) (2024)
- [Identifying and attacking the saddle point problem in high-dimensional non-convex optimization](https://mlsanity.com/paper/1406.2572/) (2014)
- [Flow Matching in Latent Space](https://mlsanity.com/paper/2307.08698/) (2023)
- [Optimal whitening and decorrelation](https://mlsanity.com/paper/1512.00809/) (2015)
- [Normalizing Flows are Capable Generative Models](https://mlsanity.com/paper/2412.06329/) (2024)
- [LAFMA: A Latent Flow Matching Model for Text-to-Audio Generation](https://mlsanity.com/paper/2406.08203/) (2024)
- [DCTRGAN: Improving the Precision of Generative Models with Reweighting](https://mlsanity.com/paper/2009.03796/) (2020)
- [Diffusion Model with Perceptual Loss](https://mlsanity.com/paper/2401.00110/) (2023)
- [Symmetrical Flow Matching: Unified Image Generation, Segmentation, and Classification with Score-Based Generative Models](https://mlsanity.com/paper/2506.10634/) (2025)
- [Score Distillation of Flow Matching Models](https://mlsanity.com/paper/2509.25127/) (2025)
- [Optimizing Noise Schedules of Generative Models in High Dimensionss](https://mlsanity.com/paper/2501.00988/) (2025)
- [Diagnosing and Improving Diffusion Models by Estimating the Optimal Loss Value](https://mlsanity.com/paper/2506.13763/) (2025)
- [CTFlow: Video-Inspired Latent Flow Matching for 3D CT Synthesis](https://mlsanity.com/paper/2508.12900/) (2025)
- [FlowLet: Conditional 3D Brain MRI Synthesis using Wavelet Flow Matching](https://mlsanity.com/paper/2601.05212/) (2026)
- [Preconditioning for Accelerated Gradient Descent Optimization and Regularization](https://mlsanity.com/paper/2410.00232/) (2024)
- [Time dependent loss reweighting for flow matching and diffusion models is theoretically justified](https://mlsanity.com/paper/2511.16599/) (2025)

## Built on by (most cited first, 2 of 2)

- [Same Flow, Different Paths: Variance Reduction in Flow Matching](https://mlsanity.com/paper/2609.17287/) (2026)
- [PMosFM: Preconditioned Manifold Matching for One-Step Physics-Constrained Generation](https://mlsanity.com/paper/2609.40287/) (2026)

## Abstract (arXiv)

Flow matching (FM) learns vector fields by regressing stochastic velocity targets along intermediate distributions pt. We identify a geometric optimization bottleneck in this regression problem: when the covariance Sigmat of pt is ill-conditioned, gradient-based training rapidly fits high-variance directions while making slow progress along low-variance ones. In an exactly solvable Gaussian setting, we prove that the excess risk is weighted by Sigmat, and that both gradient descent and stochastic gradient descent inherit condition-number-dependent convergence. We then extend the analysis to Gaussian mixtures, showing that multimodality does not average away this effect; instead, the slowest and worst-conditioned component can control optimization. Motivated by this analysis, we propose preconditioned flow matching, a precondition-then-match framework that transforms the target distribution into a more isotropic representation, trains the main flow in the transformed space, and maps generated samples back through the inverse transformation. We show theoretically that preconditioning reshapes the intermediate FM path and improves its conditioning. Across controlled Gaussian and Gaussian-mixture experiments, latent MNIST and other high resolution image datasets up to 512×512 resolution, preconditioning improves path-conditioning diagnostics, low-eigenvalue recovery, FID, MMD, precision, and recall. Compute-matched baselines and preconditioner-quality ablations further show that the gains are not explained merely by additional preconditioner parameters, but by improved geometry of the downstream flow matching problem.