mlsanityData to 7 Oct 2026

Inductive Moment Matching

arXiv 2503.07565 ↗First posted 10 Mar 2025cs.LG cs.AI stat.MLcited by 107

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Diffusion models and Flow Matching generate high-quality samples but are slow at inference, and distilling them into few-step models often leads to instability and extensive tuning. To resolve these trade-offs, we propose Inductive Moment Matching (IMM), a new class of generative models for one- or few-step sampling with a single-stage training procedure. Unlike distillation, IMM does not require pre-training initialization and optimization of two networks; and unlike Consistency Models, IMM guarantees distribution-level convergence and remains stable under various hyperparameters and standard model architectures. IMM surpasses diffusion models on ImageNet-256x256 with 1.99 FID using only 8 inference steps and achieves state-of-the-art 2-step FID of 1.98 on CIFAR-10 for a model trained from scratch.

Key facts. Cited by 107 papers in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 64 of its 69 references that are in the map. First posted on arXiv 10 Mar 2025.

Builds on

64 of 69 references.

Closest ancestors · 40

Same direction, newest first

  1. 2024One-Step Diffusion Distillation through Score Implicit Matching22 Oct
  2. 2024One Step Diffusion via Shortcut Models16 Oct
  3. 2024Simplifying, Stabilizing and Scaling Continuous-Time Consistency Models14 Oct
  4. 2024Consistency Models Made Easy20 Jun
  5. 2024Multistep Distillation of Diffusion Models via Moment Matching6 Jun
  6. 2024U-DiTs: Downsample Tokens in U-Shaped Diffusion Transformers4 May
  7. 2024Score identity Distillation: Exponentially Fast Distillation of Pretrained Diffusion Models for One-Step Generation5 Apr
  8. 2024Multistep Consistency Models11 Mar
  9. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesis5 Mar
  10. 2024SiT: Exploring Flow and Diffusion-based Generative Models with Scalable Interpolant Transformers16 Jan
  11. 2023Analyzing and Improving the Training Dynamics of Diffusion Models5 Dec
  12. 2023One-step Diffusion with Distribution Matching Distillation30 Nov
28 more
  1. 2023Adversarial Diffusion Distillation28 Nov
  2. 2023Improved Techniques for Training Consistency Models22 Oct
  3. 2023Consistency Trajectory Models: Learning Probability Flow ODE Trajectory of Diffusion1 Oct
  4. 2023PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesis30 Sep
  5. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis4 Jul
  6. 2023Restart Sampling for Improving Generative Processes26 Jun
  7. 2023Diff-Instruct: A Universal Approach for Transferring Knowledge From Pre-trained Diffusion Models29 May
  8. 2023TRACT: Denoising Diffusion Models with Transitive Closure Time-Distillation7 Mar
  9. 2023Consistency Models2 Mar
  10. 2023Understanding Diffusion Objectives as the ELBO with Simple Data Augmentation1 Mar
  11. 2023Simple diffusion: End-to-end diffusion for high resolution images26 Jan
  12. 2022Scalable Diffusion Models with Transformers19 Dec
  13. 2022Fast Sampling of Diffusion Models via Operator Learning24 Nov
  14. 2022On Distillation of Guided Diffusion Models6 Oct
  15. 2022All are Worth Words: A ViT Backbone for Diffusion Models25 Sep
  16. 2022Classifier-Free Diffusion Guidance26 Jul
  17. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps2 Jun
  18. 2022Elucidating the Design Space of Diffusion-Based Generative Models1 Jun
  19. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
  20. 2021Tackling the Generative Learning Trilemma with Denoising Diffusion GANs15 Dec
  21. 2021Variational Diffusion Models1 Jul
  22. 2021Cascaded Diffusion Models for High Fidelity Image Generation30 May
  23. 2021Diffusion Models Beat GANs on Image Synthesis11 May
  24. 2021Improved Denoising Diffusion Probabilistic Models18 Feb
  25. 2021Knowledge Distillation in Iterative Generative Models for Improved Sampling Speed7 Jan
  26. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  27. 2020Denoising Diffusion Implicit Models6 Oct
  28. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 24

  1. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  2. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  3. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  4. 2022Generative Adversarial Networkscited by 5,603
  5. 2022Flow Matching for Generative Modelingcited by 5,426
  6. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
  7. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,550
  8. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  9. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  10. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,526
  11. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,765
  12. 2020DiffWave: A Versatile Diffusion Model for Audio Synthesiscited by 1,339
12 more
  1. 2022MaskGIT: Masked Generative Image Transformercited by 1,082
  2. 2024Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Predictioncited by 1,005
  3. 2022Building Normalizing Flows with Stochastic Interpolantscited by 964
  4. 2023Stochastic Interpolants: A Unifying Framework for Flows and Diffusionscited by 897
  5. 2024WorldSimBench: Towards Video Generation Models as World Simulatorscited by 867
  6. 2024Autoregressive Image Generation without Vector Quantizationcited by 644
  7. 2020WaveGrad: Estimating Gradients for Waveform Generationcited by 620
  8. 2023AudioLDM: Text-to-Audio Generation with Latent Diffusion Modelscited by 592
  9. 2023Scaling up GANs for Text-to-Image Synthesiscited by 492
  10. 2015Generative Moment Matching Networkscited by 252
  11. 2017MMD GAN: Towards Deeper Understanding of Moment Matching Networkcited by 251
  12. 2024Physics Informed Distillation for Diffusion Modelscited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 53; 2026: 54.

107 papers.

Sort:

By area:

  1. 2025Mean Flows for One-step Generative Modelingcited by 524
  2. 2025Improved Mean Flows: On the Challenges of Fastforward Generative Modelscited by 102
  3. 2026Generative Modeling via Driftingcited by 98
  4. 2025How to build a consistency model: Learning flow maps via self-distillationcited by 96
  5. 2025Align Your Flow: Scaling Continuous-Time Flow Map Distillationcited by 81
  6. 2025Diffuse and Disperse: Image Generation with Representation Regularizationcited by 67
  7. 2026One-step Latent-free Image Generation with Pixel Mean Flowscited by 49
  8. 2025AlphaFlow: Understanding and Improving MeanFlow Modelscited by 40
  9. 2026Categorical Flow Mapscited by 37
  10. 2025Transition Models: Rethinking the Generative Learning Objectivecited by 32
  11. 2026Representation Fr'echet Loss for Visual Generationcited by 32
  12. 2025pi-Flow: Policy-Based Few-Step Generation via Imitation Distillationcited by 25
38 more
  1. 2025There is No VAE: End-to-End Pixel-Space Generative Modeling via Self-Supervised Pre-trainingcited by 23
  2. 2025Decoupled MeanFlow: Turning Flow Models into Flow Maps for Accelerated Samplingcited by 19
  3. 2025CMT: Mid-Training for Efficient Learning of Consistency, Mean Flow, and Flow Map Modelscited by 18
  4. 2026One-Step Generative Modeling via Wasserstein Gradient Flowscited by 18
  5. 2025Terminal Velocity Matchingcited by 17
  6. 2025Joint Distillation for Fast Likelihood Evaluation and Sampling in Flow-based Modelscited by 17
  7. 2025Simulating the Visual World with Artificial Intelligence: A Roadmapcited by 15
  8. 2026Gradient Flow Drifting: Generative Modeling via Wasserstein Gradient Flows of KDE-Approximated Divergencescited by 15
  9. 2025Uni-Instruct: One-step Diffusion Model through Unified Diffusion Divergence Instructioncited by 13
  10. 2025Learning an Image Editing Model without Image Editing Pairscited by 13
  11. 2025Flow Map Distillation Without Datacited by 13
  12. 2025Learn to Guide Your Diffusion Modelcited by 12
  13. 2025SoFlow: Solution Flow Models for One-Step Generative Modelingcited by 12
  14. 2025Scale-wise Distillation of Diffusion Modelscited by 11
  15. 2025Adversarial Flow Modelscited by 10
  16. 2026Stable Velocity: A Variance Perspective on Flow Matchingcited by 10
  17. 2026Representation Distribution Matching for One-Step Visual Generationcited by 10
  18. 2025One-Step Offline Distillation of Diffusion-based Models via Koopman Modelingcited by 9
  19. 2025Learning to Integrate Diffusion ODEs by Averaging the Derivativescited by 9
  20. 2025One-Step Flow Q-Learning: Addressing the Diffusion Policy Bottleneck in Offline Reinforcement Learningcited by 9
  21. 2025A Survey of Inductive Reasoning for Large Language Modelscited by 8
  22. 2025DC-Gen: Post-Training Diffusion Acceleration with Deeply Compressed Latent Spacecited by 7
  23. 2025Learning Straight Flows: Variational Flow Matching for Efficient Generationcited by 7
  24. 2025Bidirectional Normalizing Flow: From Data to Noise and Backcited by 7
  25. 2026MeanCache: From Instantaneous to Average Velocity for Accelerating Flow Matching Inferencecited by 7
  26. 2026Parallel Decoding Distillation for Fast Image and Video Generationcited by 7
  27. 2025Efficient Regression-Based Training of Normalizing Flows for Boltzmann Generatorscited by 6
  28. 2025Improved Training Technique for Shortcut Modelscited by 6
  29. 2025MeanFlow Transformers with Representation Autoencoderscited by 6
  30. 2026On the Wasserstein Gradient Flow Interpretation of Drifting Modelscited by 6
  31. 2025Soft-Di[M]O: Improving One-Step Discrete Image Generation with Soft Embeddingscited by 5
  32. 2025Scalable GANs with Transformerscited by 5
  33. 2025Neon: Negative Extrapolation From Self-Training Improves Image Generationcited by 5
  34. 2025Functional Mean Flow in Hilbert Spacecited by 5
  35. 2025Few-Step Distillation for Text-to-Image Generation: A Practical Guidecited by 5
  36. 2025B'ezierFlow: Learning B'ezier Stochastic Interpolant Schedulers for Few-Step Generationcited by 5
  37. 2026Variational Flow Maps: Make Some Noise for One-Step Conditional Generationcited by 5
  38. 2025Training Latent Diffusion Models with Interacting Particle Algorithmscited by 4

50 most cited of 107 All 107 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Fixed Point Diffusion Models
  2. 2024Flow map matching with stochastic interpolants: A mathematical framework for consistency models
  3. 2022Diffusion Models: A Comprehensive Survey of Methods and Applications
  4. 2026Simulation-free and finite-time diffusion model
  5. 2026Consistent Distribution Matching for Data-Free Diffusion Distillation
  6. 2022Progressive Distillation for Fast Sampling of Diffusion Models
  7. 2025Inference-Time Alignment of Diffusion Models via Evolutionary Algorithms
  8. 2026Adaptive Moments are Surprisingly Effective for Plug-and-Play Diffusion Sampling