mlsanityData to 14 Oct 2026

No MCMC for me: Amortized sampling for fast and stable training of energy-based models

arXiv 2010.04230 ↗First posted 8 Oct 2020cs.LG cs.AIcited by 64

Not on the map. Why?

Abstract (arXiv)

Energy-Based Models (EBMs) present a flexible and appealing way to represent uncertainty. Despite recent advances, training EBMs on high-dimensional data remains a challenging problem as the state-of-the-art approaches are costly, unstable, and require considerable tuning and domain expertise to apply successfully. In this work, we present a simple method for training EBMs at scale which uses an entropy-regularized generator to amortize the MCMC sampling typically used in EBM training. We improve upon prior MCMC-based entropy regularization methods with a fast variational approximation. We demonstrate the effectiveness of our approach by using it to train tractable likelihood models. Next, we apply our estimator to the recently proposed Joint Energy Model (JEM), where we match the original performance with faster and stable training. This allows us to extend JEM models to semi-supervised classification on tabular data from a variety of continuous domains.

Key facts. Cited by 64 papers in the map (data through 14 Oct 2026). First posted on arXiv 8 Oct 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 13; 2022: 15; 2023: 21; 2024: 6; 2025: 7; 2026: 2.

64 papers.

Sort:

By area:

  1. 2021How to Train Your Energy-Based Modelscited by 274
  2. 2021Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Modelscited by 252
  3. 2023Reduce, Reuse, Recycle: Compositional Generation with Energy-Based Diffusion Models and MCMCcited by 220
  4. 2022COLD Decoding: Energy-based Constrained Text Generation with Langevin Dynamicscited by 161
  5. 2023Motion Planning Diffusion: Learning and Planning of Robot Motions with Diffusion Modelscited by 161
  6. 2022Generative Flow Networks for Discrete Probabilistic Modelingcited by 114
  7. 2021Oops I Took A Gradient: Scalable Sampling for Discrete Distributionscited by 92
  8. 2022Your ViT is Secretly a Hybrid Discriminative-Generative Diffusion Modelcited by 58
  9. 2021Energy-Based Open-World Uncertainty Modeling for Confidence Calibrationcited by 46
  10. 2022A Tale of Two Flows: Cooperative Learning of Langevin Flow and Normalizing Flow Toward Energy-Based Modelcited by 44
  11. 2021JEM++: Improved Techniques for Training JEMcited by 28
  12. 2023Learning Data Representations with Joint Diffusion Modelscited by 22
38 more
  1. 2022Learning Implicit Priors for Motion Optimizationcited by 20
  2. 2023Learning Energy-Based Models by Cooperative Diffusion Recovery Likelihoodcited by 19
  3. 2023Guiding Energy-based Models via Contrastive Latent Variablescited by 18
  4. 2023Energy-Based Models for Anomaly Detection: A Manifold Diffusion Recovery Approachcited by 18
  5. 2021On Out-of-distribution Detection with Energy-based Modelscited by 17
  6. 2022Towards Bridging the Performance Gaps of Joint Energy-based Modelscited by 15
  7. 2022Energy-Based Contrastive Learning of Visual Representationscited by 13
  8. 2022Learning Probabilistic Models from Generator Latent Spaces with Hat EBMcited by 11
  9. 2021Bounds all around: training energy-based models with bidirectional boundscited by 10
  10. 2023Revisiting Energy Based Models as Policies: Ranking Noise Contrastive Estimation and Interpolating Energy Modelscited by 10
  11. 2021Hamiltonian Dynamics with Non-Newtonian Momentum for Rapid Samplingcited by 9
  12. 2023Training Energy-Based Models with Diffusion Contrastive Divergencescited by 9
  13. 2022Learning Energy Networks with Generalized Fenchel-Young Lossescited by 8
  14. 2023Learning Symbolic Representations Through Joint GEnerative and DIscriminative Trainingcited by 8
  15. 2021Pseudo-Spherical Contrastive Divergencecited by 7
  16. 2023Learning Energy-based Model via Dual-MCMC Teachingcited by 7
  17. 2024Improving Adversarial Energy-Based Model via Diffusion Processcited by 7
  18. 2025Follow the Energy, Find the Path: Riemannian Metrics from Energy-Based Modelscited by 7
  19. 2022Bi-level Doubly Variational Learning for Energy-based Latent Variable Modelscited by 6
  20. 2022GEDI: GEnerative and DIscriminative Training for Self-Supervised Learningcited by 6
  21. 2023M-EBM: Towards Understanding the Manifolds of Energy-Based Modelscited by 6
  22. 2025Learning Latent Energy-Based Models via Interacting Particle Langevin Dynamicscited by 5
  23. 2022Your Autoregressive Generative Model Can be Better If You Treat It as an Energy-Based Onecited by 4
  24. 2022Combating Mode Collapse in GANs via Manifold Entropy Estimationcited by 4
  25. 2023Persistently Trained, Diffusion-assisted Energy-based Modelscited by 4
  26. 2023Balanced Training of Energy-Based Models with Adaptive Flow Samplingcited by 4
  27. 2024Maximum Entropy Inverse Reinforcement Learning of Diffusion Models with Energy-Based Modelscited by 4
  28. 2021Learning Proposals for Practical Energy-Based Regressioncited by 3
  29. 2023The Triad of Failure Modes and a Possible Way Outcited by 3
  30. 2021Learning High-Dimensional Distributions with Latent Neural Fokker-Planck Kernelscited by 2
  31. 2021Conjugate Energy-Based Modelscited by 2
  32. 2023Versatile Energy-Based Probabilistic Models for High Energy Physicscited by 2
  33. 2023Latent Space Energy-based Model for Fine-grained Open Set Recognitioncited by 2
  34. 2023Unifying Self-Supervised Clustering and Energy-Based Modelscited by 2
  35. 2024Lazy Layers to Make Fine-Tuned Diffusion Models More Traceablecited by 2
  36. 2025Learning Energy-Based Generative Models via Potential Flow: A Variational Principle Approach to Probability Density Homotopy Matchingcited by 2
  37. 2021A Unified View of cGANs with and without Classifierscited by 1
  38. 2022Maximum entropy exploration in contextual bandits with neural networks and energy based modelscited by 1

50 most cited of 64 All 64 (JSON)