mlsanityData to 7 Oct 2026

Learning Latent Space Energy-Based Prior Model

arXiv 2006.08205 ↗First posted 15 Jun 2020stat.ML cs.LGcited by 104

Not on the map. Why?

Abstract (arXiv)

We propose to learn energy-based model (EBM) in the latent space of a generator model, so that the EBM serves as a prior model that stands on the top-down network of the generator model. Both the latent space EBM and the top-down network can be learned jointly by maximum likelihood, which involves short-run MCMC sampling from both the prior and posterior distributions of the latent vector. Due to the low dimensionality of the latent space and the expressiveness of the top-down network, a simple EBM in latent space can capture regularities in the data effectively, and MCMC sampling in latent space is efficient and mixes well. We show that the learned model exhibits strong performances in terms of image and text generation and anomaly detection. The one-page code can be found in supplementary materials.

Key facts. Cited by 104 papers in the map (data through 7 Oct 2026). First posted on arXiv 15 Jun 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 19; 2022: 18; 2023: 30; 2024: 18; 2025: 11; 2026: 8.

104 papers.

Sort:

By area:

  1. 2021Score-based Generative Modeling in Latent Spacecited by 634
  2. 2021Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Modelscited by 251
  3. 2022COLD Decoding: Energy-based Constrained Text Generation with Langevin Dynamicscited by 159
  4. 2021Priors in Bayesian Deep Learning: A Reviewcited by 101
  5. 2021Unleashing Transformers: Parallel Token Prediction with Discrete Absorbing Diffusion for Fast High-Resolution Image Generation from Vector-Quantized Codescited by 71
  6. 2022Latent Diffusion Energy-Based Model for Interpretable Text Modelingcited by 67
  7. 2021Controllable and Compositional Generation with Latent-Space Energy-Based Modelscited by 64
  8. 2021Trajectory Prediction with Latent Belief Energy-Based Modelcited by 60
  9. 2022Your ViT is Secretly a Hybrid Discriminative-Generative Diffusion Modelcited by 58
  10. 2021Learning Generative Vision Transformer with Energy-Based Latent Space for Saliency Predictioncited by 57
  11. 2024Deep Generative Models through the Lens of the Manifold Hypothesis: A Survey and New Connectionscited by 47
  12. 2022A Tale of Two Flows: Cooperative Learning of Langevin Flow and Normalizing Flow Toward Energy-Based Modelcited by 44
38 more
  1. 2021Data Augmentation in High Dimensional Low Sample Size Setting Using a Geometry-Based Variational Autoencodercited by 27
  2. 2021Unsupervised Foreground Extraction via Deep Region Competitioncited by 26
  3. 2022Particle algorithms for maximum likelihood training of latent variable modelscited by 26
  4. 2022Pythae: Unifying Generative Autoencoders in Python -- A Benchmarking Use Casecited by 26
  5. 2022A Geometric Perspective on Variational Autoencoderscited by 24
  6. 2023Learning Energy-Based Models by Cooperative Diffusion Recovery Likelihoodcited by 19
  7. 2022Adaptive Multi-stage Density Ratio Estimation for Learning Latent Space Energy-based Modelcited by 18
  8. 2023Diverse 3D Hand Gesture Prediction from Body Dynamics by Bilateral Hand Disentanglementcited by 18
  9. 2023Binary Latent Diffusioncited by 18
  10. 2023Energy-Based Models for Anomaly Detection: A Manifold Diffusion Recovery Approachcited by 17
  11. 2022Deep Generative model with Hierarchical Latent Factors for Time Series Anomaly Detectioncited by 16
  12. 2021Latent Space Energy-Based Model of Symbol-Vector Coupling for Text Generation and Classificationcited by 14
  13. 2024Latent Plan Transformer for Trajectory Abstraction: Planning as Latent Space Inferencecited by 14
  14. 2023Learning Energy-Based Prior Model with Diffusion-Amortized MCMCcited by 13
  15. 2023Weakly-supervised Contrastive Learning for Unsupervised Object Discoverycited by 12
  16. 2023A Review of Change of Variable Formulas for Generative Modelingcited by 12
  17. 2023Momentum Particle Maximum Likelihoodcited by 12
  18. 2021A prior-based approximate latent Riemannian metriccited by 11
  19. 2022Learning Probabilistic Models from Generator Latent Spaces with Hat EBMcited by 11
  20. 2022Energy-Based Residual Latent Transport for Unsupervised Point Cloud Completioncited by 9
  21. 2022Hierarchically Structured Task-Agnostic Continual Learningcited by 9
  22. 2023Hybrid Energy Based Model in the Feature Space for Out-of-Distribution Detectioncited by 9
  23. 2023Energy Discrepancies: A Score-Independent Loss for Energy-Based Modelscited by 9
  24. 2021MRI Reconstruction Using Deep Energy-Based Modelcited by 8
  25. 2023Learning Joint Latent Space EBM Prior Model for Multi-layer Generatorcited by 8
  26. 2023Molecule Design by Latent Space Energy-Based Modeling and Gradual Distribution Shiftingcited by 8
  27. 2023Molecule Design by Latent Prompt Transformercited by 8
  28. 2024"Noisier" Noise Contrastive Eestimation is (Almost) Maximum Likelihoodcited by 8
  29. 2021Mixture-of-Variational-Experts for Continual Learningcited by 7
  30. 2023A Tale of Two Latent Flows: Learning Latent Space Normalizing Flow with Short-run Langevin Flow for Approximate Inferencecited by 7
  31. 2023Energy-Inspired Self-Supervised Pretraining for Vision Modelscited by 7
  32. 2024Improving Adversarial Energy-Based Model via Diffusion Processcited by 7
  33. 2024Learning Multimodal Latent Generative Models with Energy-Based Priorcited by 7
  34. 2025Follow the Energy, Find the Path: Riemannian Metrics from Energy-Based Modelscited by 7
  35. 2021Elsa: Energy-based learning for semi-supervised anomaly detectioncited by 6
  36. 2022Bi-level Doubly Variational Learning for Energy-based Latent Variable Modelscited by 6
  37. 2023Diverse and Faithful Knowledge-Grounded Dialogue Generation via Sequential Posterior Inferencecited by 6
  38. 2023Learning non-Markovian Decision-Making from State-only Sequencescited by 6

50 most cited of 104 All 104 (JSON)