mlsanityData to 7 Oct 2026

Learning Non-Convergent Non-Persistent Short-Run MCMC Toward Energy-Based Model

arXiv 1904.09770 ↗First posted 22 Apr 2019stat.ML cs.LGcited by 147

Not on the map. Why?

Cited by 20 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

This paper studies a curious phenomenon in learning energy-based model (EBM) using MCMC. In each learning iteration, we generate synthesized examples by running a non-convergent, non-mixing, and non-persistent short-run MCMC toward the current model, always starting from the same initial distribution such as uniform noise distribution, and always running a fixed number of MCMC steps. After generating synthesized examples, we then update the model parameters according to the maximum likelihood learning gradient, as if the synthesized examples are fair samples from the current model. We treat this non-convergent short-run MCMC as a learned generator model or a flow model. We provide arguments for treating the learned non-convergent short-run MCMC as a valid model. We show that the learned short-run MCMC is capable of generating realistic images. More interestingly, unlike traditional EBM or MCMC, the learned short-run MCMC is capable of reconstructing observed images and interpolating between images, like generator or flow models. The code can be found in the Appendix.

Key facts. Cited by 147 papers in the map (data through 7 Oct 2026). First posted on arXiv 22 Apr 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 1; 2021: 35; 2022: 24; 2023: 46; 2024: 19; 2025: 16; 2026: 6.

147 papers.

Sort:

By area:

  1. 2022Planning with Diffusion for Flexible Behavior Synthesiscited by 1,023
  2. 20213D Shape Generation and Completion through Point-Voxel Diffusioncited by 516
  3. 2022Is Conditional Generative Modeling all you need for Decision-Making?cited by 504
  4. 2021How to Train Your Energy-Based Modelscited by 271
  5. 2021Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Modelscited by 251
  6. 2021Path Integral Sampler: a stochastic control approach for samplingcited by 175
  7. 2023AdaptDiffuser: Diffusion Models as Adaptive Self-evolving Plannerscited by 147
  8. 2021Adversarial purification with Score-based generative modelscited by 132
  9. 2022Generative Flow Networks for Discrete Probabilistic Modelingcited by 113
  10. 2022DiffuseVAE: Efficient, Controllable and High-Fidelity Generation from Low-Dimensional Latentscited by 103
  11. 2021Pixel-wise Energy-biased Abstention Learning for Anomaly Segmentation on Complex Urban Driving Scenescited by 83
  12. 2022Latent Diffusion Energy-Based Model for Interpretable Text Modelingcited by 67
38 more
  1. 2021Trajectory Prediction with Latent Belief Energy-Based Modelcited by 60
  2. 2024EM Distillation for One-step Diffusion Modelscited by 60
  3. 2021Learning Generative Vision Transformer with Energy-Based Latent Space for Saliency Predictioncited by 57
  4. 2021Energy-Based Open-World Uncertainty Modeling for Confidence Calibrationcited by 46
  5. 2022A Tale of Two Flows: Cooperative Learning of Langevin Flow and Normalizing Flow Toward Energy-Based Modelcited by 44
  6. 2020Energy-Based Imitation Learningcited by 39
  7. 2023Energy-based learning algorithms for analog computing: a comparative studycited by 36
  8. 2023Energy-Based Cross Attention for Bayesian Context Update in Text-to-Image Diffusion Modelscited by 30
  9. 2021Equilibrium and non-Equilibrium regimes in the learning of Restricted Boltzmann Machinescited by 27
  10. 2023Refining Diffusion Planner for Reliable Behavior Synthesis by Automatic Detection of Infeasible Planscited by 27
  11. 2021Autoencoding Under Normalization Constraintscited by 26
  12. 2021Towards Understanding the Generative Capability of Adversarially Robust Classifierscited by 26
  13. 2021Unsupervised Foreground Extraction via Deep Region Competitioncited by 26
  14. 2023TEA: Test-time Energy Adaptationcited by 25
  15. 2023Learning Energy-Based Models by Cooperative Diffusion Recovery Likelihoodcited by 19
  16. 2022Adaptive Multi-stage Density Ratio Estimation for Learning Latent Space Energy-based Modelcited by 18
  17. 2021On Out-of-distribution Detection with Energy-based Modelscited by 17
  18. 2023Guiding Energy-based Models via Contrastive Latent Variablescited by 17
  19. 2023Energy-Based Models for Anomaly Detection: A Manifold Diffusion Recovery Approachcited by 17
  20. 2023Efficient Training of Energy-Based Models Using Jarzynski Equalitycited by 15
  21. 2021Latent Space Energy-Based Model of Symbol-Vector Coupling for Text Generation and Classificationcited by 14
  22. 2023Energy-Based Test Sample Adaptation for Domain Generalizationcited by 14
  23. 2023Learning Energy-Based Prior Model with Diffusion-Amortized MCMCcited by 13
  24. 2023From Posterior Sampling to Meaningful Diversity in Image Restorationcited by 12
  25. 2023Time-Independent Information-Theoretic Generalization Bounds for SGLDcited by 12
  26. 2024Cascade of phase transitions in the training of Energy-based modelscited by 12
  27. 2022Learning Probabilistic Models from Generator Latent Spaces with Hat EBMcited by 11
  28. 2023Explaining the effects of non-convergent sampling in the training of Energy-Based Modelscited by 11
  29. 2021Learning Equivariant Energy Based Models with Equivariant Stein Variational Gradient Descentcited by 10
  30. 2022Learning Neural Set Functions Under the Optimal Subset Oraclecited by 10
  31. 2023Sampling Multimodal Distributions with the Vanilla Score: Benefits of Data-Based Initializationcited by 10
  32. 2024Provable Convergence and Limitations of Geometric Tempering for Langevin Dynamicscited by 10
  33. 2021Hamiltonian Dynamics with Non-Newtonian Momentum for Rapid Samplingcited by 9
  34. 2022Computed Tomography Reconstruction using Generative Energy-Based Priorscited by 9
  35. 2023Training Energy-Based Models with Diffusion Contrastive Divergencescited by 9
  36. 2023Energy Discrepancies: A Score-Independent Loss for Energy-Based Modelscited by 9
  37. 2021Learning Cycle-Consistent Cooperative Networks via Alternating MCMC Teaching for Unsupervised Cross-Domain Translationcited by 8
  38. 2021Energy-Based Generative Cooperative Saliency Predictioncited by 8

50 most cited of 147 All 147 (JSON)