mlsanityData to 7 Oct 2026

Bounds all around: training energy-based models with bidirectional bounds

arXiv 2111.00929 ↗First posted 1 Nov 2021cs.LG stat.MLcited by 10

Not on the map. Why?

Abstract (arXiv)

Energy-based models (EBMs) provide an elegant framework for density estimation, but they are notoriously difficult to train. Recent work has established links to generative adversarial networks, where the EBM is trained through a minimax game with a variational value function. We propose a bidirectional bound on the EBM log-likelihood, such that we maximize a lower bound and minimize an upper bound when solving the minimax game. We link one bound to a gradient penalty that stabilizes training, thereby providing grounding for best engineering practice. To evaluate the bounds we develop a new and efficient estimator of the Jacobi-determinant of the EBM generator. We demonstrate that these developments significantly stabilize training and yield high-quality density estimation and sample generation.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 28 of its 65 references that are in the map. First posted on arXiv 1 Nov 2021.

Builds on

28 of 65 references.

References · 28

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  3. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,362
  4. 2017Improved Training of Wasserstein GANscited by 2,223
  5. 2015Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networkscited by 2,048
  6. 2018Spectral Normalization for Generative Adversarial Networkscited by 1,618
  7. 2018Deep Anomaly Detection with Outlier Exposurecited by 939
  8. 2019Likelihood Ratios for Out-of-Distribution Detectioncited by 451
  9. 2018Assessing Generative Models via Precision and Recallcited by 401
  10. 2020Your GAN is Secretly an Energy-based Model and You Should use Discriminator Driven Latent Samplingcited by 71
  11. 2020No MCMC for me: Amortized sampling for fast and stable training of energy-based modelscited by 64
  12. 2016Cooperative Training of Descriptor and Generator Networkscited by 60
16 more
  1. 2017Gradient Estimators for Implicit Modelscited by 58
  2. 2021Hierarchical VAEs Know What They Don't Knowcited by 58
  3. 2018Learning Descriptor Networks for 3D Shape Synthesis and Analysiscited by 56
  4. 2019Maximum Entropy Generators for Energy-Based Modelscited by 55
  5. 2018A Spectral Approach to Gradient Estimation for Implicit Distributionscited by 45
  6. 2020Learning Energy-Based Model with Variational Auto-Encoder as Amortized Samplercited by 43
  7. 2018Uncertainty in the Variational Information Bottleneckcited by 39
  8. 2016Deep Directed Generative Models with Energy-Based Probability Estimationcited by 34
  9. 2018Divergence Triangle for Joint Training of Generator Model, Energy-based Model, and Inference Modelcited by 34
  10. 2021The Geometry of Deep Generative Image Models and its Applicationscited by 34
  11. 2020Regularized Autoencoders via Relaxed Injective Probability Flowcited by 23
  12. 2021A deep learning theory for neural networks grounded in physicscited by 23
  13. 2014Accurate and Conservative Estimates of MRF Log-likelihood using Reverse Annealingcited by 18
  14. 2017Calibrating Energy-based Generative Adversarial Networkscited by 13
  15. 2016Generative Adversarial Networks as Variational Training of Energy Based Modelscited by 10
  16. 2017Boltzmann machines and energy-based modelscited by 5

Built on by

Cited by year

Citing papers per year: 2022: 2; 2023: 4; 2024: 2; 2025: 2.

10 papers.

  1. 2023Energy-based learning algorithms for analog computing: a comparative studycited by 36
  2. 2024Improving Adversarial Energy-Based Model via Diffusion Processcited by 7
  3. 2022Bi-level Doubly Variational Learning for Energy-based Latent Variable Modelscited by 6
  4. 2023Learning Unnormalized Statistical Models via Compositional Optimizationcited by 5
  5. 2023Implicit Variational Inference for High-Dimensional Posteriorscited by 5
  6. 2022Combating Mode Collapse in GANs via Manifold Entropy Estimationcited by 4
  7. 2024Maximum Entropy Inverse Reinforcement Learning of Diffusion Models with Energy-Based Modelscited by 4
  8. 2023How to Construct Energy for Images? Denoising Autoencoder Can Be Energy Based Modelcited by 1
  9. 2025Learning Energy-Based Models by Self-normalising the Likelihoodcited by 0
  10. 2025Exploring bidirectional bounds for minimax-training of Energy-based modelscited by 0