mlsanityData to 7 Oct 2026

Training Latent Diffusion Models with Interacting Particle Algorithms

arXiv 2505.12412 ↗First posted 18 May 2025stat.ML cs.LGcited by 4

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

We introduce a novel particle-based algorithm for end-to-end training of latent diffusion models. We reformulate the training task as minimizing a free energy functional and obtain a gradient flow that does so. By approximating the latter with a system of interacting particles, we obtain the algorithm, which we underpin theoretically by providing error guarantees. The novel algorithm compares favorably in experiments with previous particle-based methods and variational inference analogues.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 62 of its 94 references that are in the map. First posted on arXiv 18 May 2025.

Builds on

62 of 94 references.

Closest ancestors · 23

Same direction, newest first

  1. 2025REPA-E: Unlocking VAE for End-to-End Tuning with Latent Diffusion Transformers14 Apr
  2. 2025Inductive Moment Matching10 Mar
  3. 2025VCT: Training Consistency Models with Variational Noise Coupling25 Feb
  4. 2024Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Think9 Oct
  5. 2024Learning Latent Space Hierarchical EBM Diffusion Models22 May
  6. 2024Neural Flow Diffusion Models: Learnable Forward Process for Improved Diffusion Modelling19 Apr
  7. 2023DiffEnc: Variational Diffusion with a Learned Encoder30 Oct
  8. 2023Neural Diffusion Models12 Oct
  9. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis4 Jul
  10. 2023Unifying GANs and Score-Based Diffusion as Generative Particle Models25 May
  11. 2023Consistency Models2 Mar
  12. 2023Understanding Diffusion Objectives as the ELBO with Simple Data Augmentation1 Mar
11 more
  1. 2022Scalable Diffusion Models with Transformers19 Dec
  2. 2022Maximum Likelihood Training of Implicit Nonlinear Diffusion Models27 May
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
  4. 2021Diffusion Priors In Variational Autoencoders29 Jun
  5. 2021Score-based Generative Modeling in Latent Space10 Jun
  6. 2021Diffusion Models Beat GANs on Image Synthesis11 May
  7. 2021Maximum Likelihood Training of Score-Based Diffusion Models22 Jan
  8. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  9. 2020Denoising Diffusion Implicit Models6 Oct
  10. 2020Denoising Diffusion Probabilistic Models19 Jun
  11. 2020Improved Techniques for Training Score-Based Generative Models16 Jun

Other references · 39

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  3. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  4. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  5. 2017Neural Discrete Representation Learningcited by 4,650
  6. 2014Deep Learning Face Attributes in the Wildcited by 3,679
  7. 2025Large Language Diffusion Modelscited by 872
  8. 2013Exact solutions to the nonlinear dynamics of learning in deep linear neural networkscited by 748
  9. 2020NVAE: A Deep Hierarchical Variational Autoencodercited by 635
  10. 2016Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithmcited by 498
  11. 2016Ladder Variational Autoencoderscited by 315
  12. 2019Cyclical Annealing Schedule: A Simple Approach to Mitigating KL Vanishingcited by 191
27 more
  1. 2017Optimizing the Latent Space of Generative Networkscited by 157
  2. 2019Learning Non-Convergent Non-Persistent Short-Run MCMC Toward Energy-Based Modelcited by 147
  3. 2019Maximum Mean Discrepancy Gradient Flowcited by 123
  4. 2015Preconditioned Stochastic Gradient Langevin Dynamics for Deep Neural Networkscited by 121
  5. 2020Learning Latent Space Energy-Based Prior Modelcited by 104
  6. 2022On the SDEs and Scaling Rules for Adaptive Gradient Algorithmscited by 100
  7. 2022Variational inference via Wasserstein gradient flowscited by 84
  8. 2019On the geometry of Stein variational gradient descentcited by 82
  9. 2023Forward-backward Gaussian variational inference via JKO in the Bures-Wasserstein Spacecited by 36
  10. 2022Particle algorithms for maximum likelihood training of latent variable modelscited by 26
  11. 2020MCMC Should Mix: Learning Energy-Based Model with Neural Transport Latent Space MCMCcited by 25
  12. 2024Implicit Diffusion: Efficient Optimization through Stochastic Samplingcited by 22
  13. 2024Deep MMD Gradient Flow without adversarial trainingcited by 18
  14. 2023Interacting Particle Langevin Algorithm for Maximum Marginal Likelihood Estimationcited by 17
  15. 2022Diffusion bridges vector quantized Variational AutoEncoderscited by 16
  16. 2020Generalization Properties of Optimal Transport GANs with Latent Distribution Learningcited by 13
  17. 2023Learning Energy-Based Prior Model with Diffusion-Amortized MCMCcited by 13
  18. 2024Error bounds for particle gradient descent, and extensions of the log-Sobolev and Talagrand inequalitiescited by 13
  19. 2020Stochastic Gradient Langevin Dynamics Algorithms with Adaptive Driftscited by 12
  20. 2023MonoFlow: Rethinking Divergence GANs via the Perspective of Wasserstein Gradient Flowscited by 12
  21. 2023Momentum Particle Maximum Likelihoodcited by 12
  22. 2023Tuning-Free Maximum Likelihood Training of Latent Variable Models via Coin Bettingcited by 11
  23. 2024Proximal Interacting Particle Langevin Algorithmscited by 8
  24. 2024Hierarchical VAE with a Diffusion-based VampPriorcited by 8
  25. 2024Particle Semi-Implicit Variational Inferencecited by 7
  26. 2024Kinetic Interacting Particle Langevin Monte Carlocited by 7
  27. 2022GANs as Gradient Flows that Convergecited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 2.

4 papers.

  1. 2025Learning Latent Variable Models via Jarzynski-adjusted Langevin Algorithmcited by 3
  2. 2025Learning Latent Energy-Based Models via Interacting Particle Langevin Dynamicscited by 3
  3. 2026Momentum SVGD-EM for Accelerated Maximum Marginal Likelihood Estimationcited by 0
  4. 2026Particle-based Generalised Stochastic Optimisationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Diffusion Models: A Mathematical Introduction
  2. 2025Scalable Discrete Diffusion Samplers: Combinatorial Optimization and Statistical Physics
  3. 2025Combining complex Langevin dynamics with score-based and energy-based diffusion models
  4. 2026The Diffusion Encoder
  5. 2025Diffusion As Self-Distillation: End-to-End Latent Diffusion In One Model
  6. 2025AccuQuant: Simulating Multiple Denoising Steps for Quantizing Diffusion Models
  7. 2024Improved off-policy training of diffusion samplers
  8. 2025Accelerating Diffusion Model Training under Minimal Budgets: A Condensation-Based Perspective