# Autoregressive Energy Machines

arXiv: https://arxiv.org/abs/1904.05626 · HTML: https://mlsanity.com/paper/1904.05626/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Apr 2019.

## Built on by (most cited first, 23 of 23)

- [Implicit Behavioral Cloning](https://mlsanity.com/paper/2109.00137/) (2021)
- [Flow Network based Generative Models for Non-Iterative Diverse Candidate Generation](https://mlsanity.com/paper/2106.04399/) (2021)
- [GFlowNet Foundations](https://mlsanity.com/paper/2111.09266/) (2021)
- [Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Models](https://mlsanity.com/paper/2103.04922/) (2021)
- [Copula Flows for Synthetic Data Generation](https://mlsanity.com/paper/2101.00598/) (2021)
- [Autoencoding Under Normalization Constraints](https://mlsanity.com/paper/2105.05735/) (2021)
- [Arbitrary Conditional Distributions with Energy](https://mlsanity.com/paper/2102.04426/) (2021)
- [A Unified Deep Model of Learning from both Data and Queries for Cardinality Estimation](https://mlsanity.com/paper/2107.12295/) (2021)
- [Neural Recursive Belief States in Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2102.02274/) (2021)
- [Squared Neural Families: A New Class of Tractable Density Models](https://mlsanity.com/paper/2305.13552/) (2023)
- [Learning Energy Networks with Generalized Fenchel-Young Losses](https://mlsanity.com/paper/2205.09589/) (2022)
- [The autoregressive neural network architecture of the Boltzmann distribution of pairwise interacting spins systems](https://mlsanity.com/paper/2302.08347/) (2023)
- [Learning Joint Latent Space EBM Prior Model for Multi-layer Generator](https://mlsanity.com/paper/2306.06323/) (2023)
- [Probabilistic Integral Circuits](https://mlsanity.com/paper/2310.16986/) (2023)
- [Learning Unnormalized Statistical Models via Compositional Optimization](https://mlsanity.com/paper/2306.07485/) (2023)
- [Lagrangian Flow Networks for Conservation Laws](https://mlsanity.com/paper/2305.16846/) (2023)
- [Posterior Matching for Arbitrary Conditioning](https://mlsanity.com/paper/2201.12414/) (2022)
- [Your Autoregressive Generative Model Can be Better If You Treat It as an Energy-Based One](https://mlsanity.com/paper/2206.12840/) (2022)
- [Neural Inverse Transform Sampler](https://mlsanity.com/paper/2206.11172/) (2022)
- [Autoregressive Generative Modeling with Noise Conditional Maximum Likelihood Estimation](https://mlsanity.com/paper/2210.10715/) (2022)
- [On the connection between Noise-Contrastive Estimation and Contrastive Divergence](https://mlsanity.com/paper/2402.16688/) (2024)
- [Energy-Based Model for Accurate Estimation of Shapley Values in Feature Attribution](https://mlsanity.com/paper/2404.01078/) (2024)
- [DistJoin: A Decoupled Join Cardinality Estimator based on Adaptive Neural Predicate Modulation](https://mlsanity.com/paper/2503.08994/) (2025)

## Abstract (arXiv)

Neural density estimators are flexible families of parametric models which have seen widespread use in unsupervised machine learning in recent years. Maximum-likelihood training typically dictates that these models be constrained to specify an explicit density. However, this limitation can be overcome by instead using a neural network to specify an energy function, or unnormalized density, which can subsequently be normalized to obtain a valid distribution. The challenge with this approach lies in accurately estimating the normalizing constant of the high-dimensional energy function. We propose the Autoregressive Energy Machine, an energy-based model which simultaneously learns an unnormalized density and computes an importance-sampling estimate of the normalizing constant for each conditional in an autoregressive decomposition. The Autoregressive Energy Machine achieves state-of-the-art performance on a suite of density-estimation tasks.