# Energy-Inspired Models: Learning with Sampler-Induced Distributions

arXiv: https://arxiv.org/abs/1910.14265 · HTML: https://mlsanity.com/paper/1910.14265/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Oct 2019.

## Built on by (most cited first, 12 of 12)

- [Score identity Distillation: Exponentially Fast Distillation of Pretrained Diffusion Models for One-Step Generation](https://mlsanity.com/paper/2404.04057/) (2024)
- [Energy-guided Entropic Neural Optimal Transport](https://mlsanity.com/paper/2304.06094/) (2023)
- [SIXO: Smoothing Inference with Twisted Objectives](https://mlsanity.com/paper/2206.05952/) (2022)
- [BR-SNIS: Bias Reduced Self-Normalized Importance Sampling](https://mlsanity.com/paper/2207.06364/) (2022)
- [Energy-Guided Continuous Entropic Barycenter Estimation for General Costs](https://mlsanity.com/paper/2310.01105/) (2023)
- [EigenVI: score-based variational inference with orthogonal function expansions](https://mlsanity.com/paper/2410.24054/) (2024)
- [NAS-X: Neural Adaptive Smoothing via Twisting](https://mlsanity.com/paper/2308.14864/) (2023)
- [Reparameterized Variational Rejection Sampling](https://mlsanity.com/paper/2309.14612/) (2023)
- [Learning Energy-based Variational Latent Prior for VAEs](https://mlsanity.com/paper/2510.00260/) (2025)
- [NEO: Non Equilibrium Sampling on the Orbit of a Deterministic Transform](https://mlsanity.com/paper/2103.10943/) (2021)
- [A stepped sampling method for video detection using LSTM](https://mlsanity.com/paper/2107.08471/) (2021)
- [Modeling Structure with Undirected Neural Networks](https://mlsanity.com/paper/2202.03760/) (2022)

## Abstract (arXiv)

Energy-based models (EBMs) are powerful probabilistic models, but suffer from intractable sampling and density evaluation due to the partition function. As a result, inference in EBMs relies on approximate sampling algorithms, leading to a mismatch between the model and inference. Motivated by this, we consider the sampler-induced distribution as the model of interest and maximize the likelihood of this model. This yields a class of energy-inspired models (EIMs) that incorporate learned energy functions while still providing exact samples and tractable log-likelihood lower bounds. We describe and evaluate three instantiations of such models based on truncated rejection sampling, self-normalized importance sampling, and Hamiltonian importance sampling. These models outperform or perform comparably to the recently proposed Learned Accept/Reject Sampling algorithm and provide new insights on ranking Noise Contrastive Estimation and Contrastive Predictive Coding. Moreover, EIMs allow us to generalize a recent connection between multi-sample variational lower bounds and auxiliary variable variational inference. We show how recent variational bounds can be unified with EIMs as the variational family.