mlsanityData to 14 Oct 2026

Maximum Entropy Generators for Energy-Based Models

arXiv 1901.08508 ↗First posted 24 Jan 2019cs.LG cs.AI stat.MLcited by 56

Not on the map. Why?

Abstract (arXiv)

Maximum likelihood estimation of energy-based models is a challenging problem due to the intractability of the log-likelihood gradient. In this work, we propose learning both the energy function and an amortized approximate sampling mechanism using a neural generator network, which provides an efficient approximation of the log-likelihood gradient. The resulting objective requires maximizing entropy of the generated samples, which we perform using recently proposed nonparametric mutual information estimators. Finally, to stabilize the resulting adversarial game, we use a zero-centered gradient penalty derived as a necessary condition from the score matching literature. The proposed technique can generate sharp images with Inception and FID scores competitive with recent GAN techniques, does not suffer from mode collapse, and is competitive with state-of-the-art anomaly detection techniques.

Key facts. Cited by 56 papers in the map (data through 14 Oct 2026). First posted on arXiv 24 Jan 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2019: 1; 2020: 0; 2021: 19; 2022: 6; 2023: 23; 2024: 4; 2025: 2; 2026: 1.

56 papers.

Sort:

By area:

  1. 2022Diffusion Models: A Comprehensive Survey of Methods and Applicationscited by 1,114
  2. 2021GFlowNet Foundationscited by 290
  3. 2021How to Train Your Energy-Based Modelscited by 274
  4. 2021Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Modelscited by 252
  5. 2021Spatio-temporal Self-Supervised Representation Learning for 3D Point Cloudscited by 180
  6. 2023Generative Time Series Forecasting with Diffusion, Denoise, and Disentanglementcited by 90
  7. 2019Imitation Learning from Observations by Minimizing Inverse Dynamics Disagreementcited by 46
  8. 2021Towards Understanding the Generative Capability of Adversarially Robust Classifierscited by 26
  9. 2023Energy-guided Entropic Neural Optimal Transportcited by 26
  10. 2022Learning Implicit Priors for Motion Optimizationcited by 20
  11. 2023Learning Energy-Based Models by Cooperative Diffusion Recovery Likelihoodcited by 19
  12. 2022Adaptive Multi-stage Density Ratio Estimation for Learning Latent Space Energy-based Modelcited by 18
38 more
  1. 2023Guiding Energy-based Models via Contrastive Latent Variablescited by 18
  2. 2023Energy-Based Models for Anomaly Detection: A Manifold Diffusion Recovery Approachcited by 18
  3. 2023EGC: Image Generation and Classification via a Diffusion Energy-Based Modelcited by 16
  4. 2023A survey and taxonomy of loss functions in machine learningcited by 15
  5. 2023Learning Energy-Based Prior Model with Diffusion-Amortized MCMCcited by 13
  6. 2021MemStream: Memory-Based Streaming Anomaly Detectioncited by 12
  7. 2023Explaining the effects of non-convergent sampling in the training of Energy-Based Modelscited by 12
  8. 2022Learning Probabilistic Models from Generator Latent Spaces with Hat EBMcited by 11
  9. 2021Learning Equivariant Energy Based Models with Equivariant Stein Variational Gradient Descentcited by 10
  10. 2021Bounds all around: training energy-based models with bidirectional boundscited by 10
  11. 2023Revisiting Energy Based Models as Policies: Ranking Noise Contrastive Estimation and Interpolating Energy Modelscited by 10
  12. 2021Hamiltonian Dynamics with Non-Newtonian Momentum for Rapid Samplingcited by 9
  13. 2022Non-Imaging Medical Data Synthesis for Trustworthy AI: A Comprehensive Surveycited by 9
  14. 2023Energy Discrepancies: A Score-Independent Loss for Energy-Based Modelscited by 9
  15. 2023Time-series Generation by Contrastive Imitationcited by 9
  16. 2021MRI Reconstruction Using Deep Energy-Based Modelcited by 8
  17. 2023Learning Joint Latent Space EBM Prior Model for Multi-layer Generatorcited by 8
  18. 2023Molecule Design by Latent Space Energy-Based Modeling and Gradual Distribution Shiftingcited by 8
  19. 2024"Noisier" Noise Contrastive Eestimation is (Almost) Maximum Likelihoodcited by 8
  20. 2021Pseudo-Spherical Contrastive Divergencecited by 7
  21. 2023A Tale of Two Latent Flows: Learning Latent Space Normalizing Flow with Short-run Langevin Flow for Approximate Inferencecited by 7
  22. 2023Learning Energy-based Model via Dual-MCMC Teachingcited by 7
  23. 2024Improving Adversarial Energy-Based Model via Diffusion Processcited by 7
  24. 2023Learning Hierarchical Features with Joint Latent Space Energy-Based Priorcited by 6
  25. 2023Implicit Variational Inference for High-Dimensional Posteriorscited by 5
  26. 2022Combating Mode Collapse in GANs via Manifold Entropy Estimationcited by 4
  27. 2023Neural Sampling in Hierarchical Exponential-family Energy-based Modelscited by 4
  28. 2024Maximum Entropy Inverse Reinforcement Learning of Diffusion Models with Energy-Based Modelscited by 4
  29. 2021Learning Proposals for Practical Energy-Based Regressioncited by 3
  30. 2021Learning High-Dimensional Distributions with Latent Neural Fokker-Planck Kernelscited by 2
  31. 2021GraphAnoGAN: Detecting Anomalous Snapshots from Attributed Graphscited by 2
  32. 2023Learning Energy-Based Representations of Quantum Many-Body Statescited by 2
  33. 2021Unsupervised Learning of Multi-level Structures for Anomaly Detectioncited by 1
  34. 2021A Unified View of cGANs with and without Classifierscited by 1
  35. 2023How to Construct Energy for Images? Denoising Autoencoder Can Be Energy Based Modelcited by 1
  36. 2023Progressive Energy-Based Cooperative Learning for Multi-Domain Image-to-Image Translationcited by 1
  37. 2021Deep Consensus Learningcited by 0
  38. 2021Deep Probabilistic Graphical Modelingcited by 0

50 most cited of 56 All 56 (JSON)