mlsanityData to 7 Oct 2026

How to Construct Energy for Images? Denoising Autoencoder Can Be Energy Based Model

arXiv 2303.03887 ↗First posted 5 Mar 2023cs.CV cs.LGcited by 1

Not on the map. Why?

Abstract (arXiv)

Energy-based models parameterize the unnormalized log-probability of data samples, but there is a lack of guidance on how to construct the "energy". In this paper, we propose a Denoising-EBM which decomposes the image energy into "semantic energy" and "texture energy". We define the "semantic energy" in the latent space of DAE to model the high-level representations, and define the pixel-level reconstruction error for denoising as "texture energy". Inspired by score-based model, our model utilizes multi-scale noisy samples for maximum-likelihood training and it outputs a vector instead of a scalar for exploring a larger set of functions during optimization. After training, the semantics are first synthesized by fast MCMC through "semantic energy", and then the pixel-level refinement of semantic image will be performed to generate perfect samples based on "texture energy". Ultimately, our model can outperform most EBMs in image generation. And we also demonstrate that Denoising-EBM has top performance among EBMs for out-of-distribution detection.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 38 of its 49 references that are in the map. First posted on arXiv 5 Mar 2023.

Builds on

38 of 49 references.

References · 38

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  4. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  5. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  6. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  7. 2014Deep Learning Face Attributes in the Wildcited by 3,679
  8. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
  9. 2016Improved Techniques for Training GANscited by 2,938
  10. 2017Improved Training of Wasserstein GANscited by 2,223
  11. 2016Identity Mappings in Deep Residual Networkscited by 1,960
  12. 2018Spectral Normalization for Generative Adversarial Networkscited by 1,618
26 more
  1. 2018Glow: Generative Flow with Invertible 1x1 Convolutionscited by 1,539
  2. 2016Pixel Recurrent Neural Networkscited by 814
  3. 2020NVAE: A Deep Hierarchical Variational Autoencodercited by 635
  4. 2019Likelihood Ratios for Out-of-Distribution Detectioncited by 451
  5. 2019Your Classifier is Secretly an Energy Based Model and You Should Treat it Like Onecited by 402
  6. 2021StyleGAN-V: A Continuous Video Generator with the Price, Image Quality and Perks of StyleGAN2cited by 300
  7. 2020Likelihood Regret: An Out-of-Distribution Detection Score For Variational Auto-encodercited by 145
  8. 2020Learning Energy-Based Models by Diffusion Recovery Likelihoodcited by 122
  9. 2016Energy-based Generative Adversarial Networkcited by 119
  10. 2013Generalized Denoising Auto-Encoders as Generative Modelscited by 113
  11. 2020Learning Latent Space Energy-Based Prior Modelcited by 104
  12. 2020VAEBM: A Symbiosis between Variational Autoencoders and Energy-based Modelscited by 103
  13. 2020Generalized Energy Based Modelscited by 73
  14. 2020No MCMC for me: Amortized sampling for fast and stable training of energy-based modelscited by 64
  15. 2016Cooperative Training of Descriptor and Generator Networkscited by 60
  16. 2019Maximum Entropy Generators for Energy-Based Modelscited by 55
  17. 2020Learning Energy-Based Model with Variational Auto-Encoder as Amortized Samplercited by 43
  18. 2019AlignFlow: Cycle Consistent Learning from Multiple Domains via Normalizing Flowscited by 33
  19. 2020Joint Training of Variational Auto-Encoder and Latent Energy-Based Modelcited by 30
  20. 2019Learning Energy-Based Models in High-Dimensional Spaces with Multi-scale Denoising Score Matchingcited by 26
  21. 2021Autoencoding Under Normalization Constraintscited by 26
  22. 2020Hybrid Discriminative-Generative Training via Contrastive Learningcited by 24
  23. 2019Learning Representations by Maximizing Mutual Information in Variational Autoencoderscited by 20
  24. 2020Conditional Coupled Generative Adversarial Networks for Zero-Shot Domain Adaptationcited by 10
  25. 2021Bounds all around: training energy-based models with bidirectional boundscited by 10
  26. 2022Bi-level Doubly Variational Learning for Energy-based Latent Variable Modelscited by 6

Built on by

Cited by year

Citing papers per year: 2024: 1.

1 paper.

  1. 2024On gauge freedom, conservativity and intrinsic dimensionality estimation in diffusion modelscited by 21

Similar, not cited

Close in wording, no citation either way

  1. 2025Learning normalized image densities via dual score matching
  2. 2023Learning Energy-Based Models by Cooperative Diffusion Recovery Likelihood
  3. 2023Learning Gradually Non-convex Image Priors Using Score Matching
  4. 2026A Diffusive Classification Loss for Learning Energy-based Generative Models
  5. 2026Learning Energy-Based Models from Stochastic Interpolants using Spatiotemporal Differences
  6. 2026Annealed Langevin Posterior Sampling (ALPS): A Rapid Algorithm for Image Restoration with Multiscale Energy Models
  7. 2026Learning Normalized Energy Models for Linear Inverse Problems
  8. 2022EGSDE: Unpaired Image-to-Image Translation via Energy-Guided Stochastic Differential Equations