mlsanityData to 7 Oct 2026

Learning Probabilistic Models from Generator Latent Spaces with Hat EBM

arXiv 2210.16486 ↗First posted 29 Oct 2022cs.CV cs.LG stat.MLcited by 11

Not on the map. Why?

Abstract (arXiv)

This work proposes a method for using any generator network as the foundation of an Energy-Based Model (EBM). Our formulation posits that observed images are the sum of unobserved latent variables passed through the generator network and a residual random variable that spans the gap between the generator output and the image manifold. One can then define an EBM that includes the generator as part of its forward pass, which we call the Hat EBM. The model can be trained without inferring the latent variables of the observed data or calculating the generator Jacobian determinant. This enables explicit probabilistic modeling of the output distribution of any type of generator network. Experiments show strong performance of the proposed method on (1) unconditional ImageNet synthesis at 128x128 resolution, (2) refining the output of existing generators, and (3) learning EBMs that incorporate non-probabilistic generators. Code and pretrained models to reproduce our results are available at https://github.com/point0bar1/hat-ebm.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 34 of its 53 references that are in the map. First posted on arXiv 29 Oct 2022.

Builds on

34 of 53 references.

References · 34

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2022Generative Adversarial Networkscited by 5,603
  3. 2013Auto-Encoding Variational Bayescited by 3,980
  4. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  5. 2015Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networkscited by 2,048
  6. 2018Spectral Normalization for Generative Adversarial Networkscited by 1,618
  7. 2020Energy-based Out-of-distribution Detectioncited by 1,296
  8. 2020Improved Techniques for Training Score-Based Generative Modelscited by 1,036
  9. 2018Do Deep Generative Models Know What They Don't Know?cited by 443
  10. 2019Your Classifier is Secretly an Energy Based Model and You Should Treat it Like Onecited by 402
  11. 2016A Theory of Generative ConvNetcited by 174
  12. 2019Learning Non-Convergent Non-Persistent Short-Run MCMC Toward Energy-Based Modelcited by 147
22 more
  1. 2020Improved Contrastive Divergence Training of Energy Based Modelscited by 127
  2. 2019On the Anatomy of MCMC-Based Maximum Likelihood Learning of Energy-Based Modelscited by 105
  3. 2020Learning Latent Space Energy-Based Prior Modelcited by 104
  4. 2020VAEBM: A Symbiosis between Variational Autoencoders and Energy-based Modelscited by 103
  5. 2018Self-Supervised GANs via Auxiliary Rotation Losscited by 94
  6. 2019Flow Contrastive Estimation of Energy-Based Modelscited by 75
  7. 2020Generalized Energy Based Modelscited by 73
  8. 2020Your GAN is Secretly an Energy-based Model and You Should use Discriminator Driven Latent Samplingcited by 71
  9. 2020No MCMC for me: Amortized sampling for fast and stable training of energy-based modelscited by 64
  10. 2020Stochastic Security: Adversarial Defense Using Long-Run Dynamics of Energy-Based Modelscited by 58
  11. 2018Learning Descriptor Networks for 3D Shape Synthesis and Analysiscited by 56
  12. 2019Maximum Entropy Generators for Energy-Based Modelscited by 55
  13. 2020Refining Deep Generative Models via Discriminator Gradient Flowcited by 45
  14. 2020InfoMax-GAN: Improved Adversarial Image Generation via Information Maximization and Contrastive Learningcited by 37
  15. 2016Deep Directed Generative Models with Energy-Based Probability Estimationcited by 34
  16. 2018Divergence Triangle for Joint Training of Generator Model, Energy-based Model, and Inference Modelcited by 34
  17. 2017Learning Energy-Based Models as Generative ConvNets via Multi-grid Modeling and Samplingcited by 32
  18. 2019Discriminator optimal transportcited by 25
  19. 2020Hybrid Discriminative-Generative Training via Contrastive Learningcited by 24
  20. 2020Mimicry: Towards the Reproducibility of GAN Researchcited by 14
  21. 2021Latent Space Energy-Based Model of Symbol-Vector Coupling for Text Generation and Classificationcited by 14
  22. 2017Calibrating Energy-based Generative Adversarial Networkscited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 4; 2024: 4; 2025: 2; 2026: 1.

11 papers.

  1. 2024You Only Sample Once: Taming One-Step Text-to-Image Synthesis by Self-Cooperative Diffusion GANscited by 25
  2. 2023Learning Energy-Based Models by Cooperative Diffusion Recovery Likelihoodcited by 19
  3. 2023EGC: Image Generation and Classification via a Diffusion Energy-Based Modelcited by 16
  4. 2024Improving Adversarial Energy-Based Model via Diffusion Processcited by 7
  5. 2023Learning Energy-based Model via Dual-MCMC Teachingcited by 6
  6. 2024Learning Latent Space Hierarchical EBM Diffusion Modelscited by 6
  7. 2025Outlier Synthesis via Hamiltonian Monte Carlo for Out-of-Distribution Detectioncited by 4
  8. 2023Energy-Calibrated VAE with Test Time Free Lunchcited by 3
  9. 2025Learning Energy-Based Generative Models via Potential Flow: A Variational Principle Approach to Probability Density Homotopy Matchingcited by 2
  10. 2024Variational Potential Flow: A Novel Probabilistic Framework for Energy-Based Generative Modellingcited by 0
  11. 2026Annealed Langevin Posterior Sampling (ALPS): A Rapid Algorithm for Image Restoration with Multiscale Energy Modelscited by 0