# Deep Directed Generative Models with Energy-Based Probability Estimation

arXiv: https://arxiv.org/abs/1606.03439 · HTML: https://mlsanity.com/paper/1606.03439/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Jun 2016.

## Built on by (most cited first, 30 of 34)

- [Diffusion Models: A Comprehensive Survey of Methods and Applications](https://mlsanity.com/paper/2209.00796/) (2022)
- [How to Train Your Energy-Based Models](https://mlsanity.com/paper/2101.03288/) (2021)
- [Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Models](https://mlsanity.com/paper/2103.04922/) (2021)
- [Learning to Compose Visual Relations](https://mlsanity.com/paper/2111.09297/) (2021)
- [Unsupervised Learning of Compositional Energy Concepts](https://mlsanity.com/paper/2111.03042/) (2021)
- [Energy-Based Open-World Uncertainty Modeling for Confidence Calibration](https://mlsanity.com/paper/2107.12628/) (2021)
- [Stochastic Physics-Informed Neural Ordinary Differential Equations](https://mlsanity.com/paper/2109.01621/) (2021)
- [Learning Energy-Based Models by Cooperative Diffusion Recovery Likelihood](https://mlsanity.com/paper/2309.05153/) (2023)
- [EGC: Image Generation and Classification via a Diffusion Energy-Based Model](https://mlsanity.com/paper/2304.02012/) (2023)
- [Explaining the effects of non-convergent sampling in the training of Energy-Based Models](https://mlsanity.com/paper/2301.09428/) (2023)
- [Learning Probabilistic Models from Generator Latent Spaces with Hat EBM](https://mlsanity.com/paper/2210.16486/) (2022)
- [EigenVI: score-based variational inference with orthogonal function expansions](https://mlsanity.com/paper/2410.24054/) (2024)
- [Bounds all around: training energy-based models with bidirectional bounds](https://mlsanity.com/paper/2111.00929/) (2021)
- [Time-series Generation by Contrastive Imitation](https://mlsanity.com/paper/2311.01388/) (2023)
- [Molecule Design by Latent Space Energy-Based Modeling and Gradual Distribution Shifting](https://mlsanity.com/paper/2306.14902/) (2023)
- [Moment Matching Denoising Gibbs Sampling](https://mlsanity.com/paper/2305.11650/) (2023)
- [Your Autoregressive Generative Model Can be Better If You Treat It as an Energy-Based One](https://mlsanity.com/paper/2206.12840/) (2022)
- [Persistently Trained, Diffusion-assisted Energy-based Models](https://mlsanity.com/paper/2304.10707/) (2023)
- [Neural Sampling in Hierarchical Exponential-family Energy-based Models](https://mlsanity.com/paper/2310.08431/) (2023)
- [Learning Proposals for Practical Energy-Based Regression](https://mlsanity.com/paper/2110.11948/) (2021)
- [POSEIDON: Physics-Optimized Seismic Energy Inference and Detection Operating Network](https://mlsanity.com/paper/2601.02264/) (2026)
- [Learning Energy-Based Representations of Quantum Many-Body States](https://mlsanity.com/paper/2304.04058/) (2023)
- [TERM Model: Tensor Ring Mixture Model for Density Estimation](https://mlsanity.com/paper/2312.08075/) (2023)
- [A Retrospect to Multi-prompt Learning across Vision and Language](https://mlsanity.com/paper/2511.00191/) (2025)
- [Progressive Energy-Based Cooperative Learning for Multi-Domain Image-to-Image Translation](https://mlsanity.com/paper/2306.14448/) (2023)
- [Adversarial Likelihood Estimation With One-Way Flows](https://mlsanity.com/paper/2307.09882/) (2023)
- [Deep Consensus Learning](https://mlsanity.com/paper/2103.08475/) (2021)
- [EBM Life Cycle: MCMC Strategies for Synthesis, Defense, and Density Modeling](https://mlsanity.com/paper/2205.12243/) (2022)
- [Kolmogorov-Smirnov GAN](https://mlsanity.com/paper/2406.19948/) (2024)
- [Dataset Augmentation by Mixing Visual Concepts](https://mlsanity.com/paper/2412.15358/) (2024)

## Abstract (arXiv)

Training energy-based probabilistic models is confronted with apparently intractable sums, whose Monte Carlo estimation requires sampling from the estimated probability distribution in the inner loop of training. This can be approximately achieved by Markov chain Monte Carlo methods, but may still face a formidable obstacle that is the difficulty of mixing between modes with sharp concentrations of probability. Whereas an MCMC process is usually derived from a given energy function based on mathematical considerations and requires an arbitrarily long time to obtain good and varied samples, we propose to train a deep directed generative model (not a Markov chain) so that its sampling distribution approximately matches the energy function that is being trained. Inspired by generative adversarial networks, the proposed framework involves training of two models that represent dual views of the estimated probability distribution: the energy function (mapping an input configuration to a scalar energy value) and the generator (mapping a noise vector to a generated configuration), both represented by deep neural networks.