# STANLEY: Stochastic Gradient Anisotropic Langevin Dynamics for Learning Energy-Based Models

arXiv: https://arxiv.org/abs/2310.12667 · HTML: https://mlsanity.com/paper/2310.12667/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 39 of its 89 references that are in the map.
- First posted on arXiv 19 Oct 2023.

## Builds on (most cited first, 30 of 39)

- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Energy-based Out-of-distribution Detection](https://mlsanity.com/paper/2010.03759/) (2020)
- [Reinforcement Learning with Deep Energy-Based Policies](https://mlsanity.com/paper/1702.08165/) (2017)
- [Sliced Score Matching: A Scalable Approach to Density and Score Estimation](https://mlsanity.com/paper/1905.07088/) (2019)
- [A Theory of Generative ConvNet](https://mlsanity.com/paper/1602.03264/) (2016)
- [Learning Non-Convergent Non-Persistent Short-Run MCMC Toward Energy-Based Model](https://mlsanity.com/paper/1904.09770/) (2019)
- [Improved Contrastive Divergence Training of Energy Based Models](https://mlsanity.com/paper/2012.01316/) (2020)
- [On the Anatomy of MCMC-Based Maximum Likelihood Learning of Energy-Based Models](https://mlsanity.com/paper/1903.12370/) (2019)
- [Learning Latent Space Energy-Based Prior Model](https://mlsanity.com/paper/2006.08205/) (2020)
- [Residual Energy-Based Models for Text Generation](https://mlsanity.com/paper/2004.11714/) (2020)
- [Flow Contrastive Estimation of Energy-Based Models](https://mlsanity.com/paper/1912.00589/) (2019)
- [Cooperative Training of Descriptor and Generator Networks](https://mlsanity.com/paper/1609.09408/) (2016)
- [Generative PointNet: Deep Energy-Based Learning on Unordered Point Sets for 3D Generation, Reconstruction and Classification](https://mlsanity.com/paper/2004.01301/) (2020)
- [Learning Generative Vision Transformer with Energy-Based Latent Space for Saliency Prediction](https://mlsanity.com/paper/2112.13528/) (2021)
- [Learning Descriptor Networks for 3D Shape Synthesis and Analysis](https://mlsanity.com/paper/1804.00586/) (2018)
- [A Tale of Two Flows: Cooperative Learning of Langevin Flow and Normalizing Flow Toward Energy-Based Model](https://mlsanity.com/paper/2205.06924/) (2022)
- [Learning Energy-Based Model with Variational Auto-Encoder as Amortized Sampler](https://mlsanity.com/paper/2012.14936/) (2020)
- [Learning deep kernels for exponential family densities](https://mlsanity.com/paper/1811.08357/) (2018)
- [Divergence Triangle for Joint Training of Generator Model, Energy-based Model, and Inference Model](https://mlsanity.com/paper/1812.10907/) (2018)
- [Learning Energy-Based Models as Generative ConvNets via Multi-grid Modeling and Sampling](https://mlsanity.com/paper/1709.08868/) (2017)
- [The Tamed Unadjusted Langevin Algorithm](https://mlsanity.com/paper/1710.05559/) (2017)
- [Joint Training of Variational Auto-Encoder and Latent Energy-Based Model](https://mlsanity.com/paper/2006.06059/) (2020)
- [Training Deep Energy-Based Models with f-Divergence Minimization](https://mlsanity.com/paper/2003.03463/) (2020)
- [Learning Noise-Aware Encoder-Decoder from Noisy Labels by Alternating Back-Propagation for Saliency Detection](https://mlsanity.com/paper/2007.12211/) (2020)
- [Generative VoxelNet: Learning Energy-Based Models for 3D Shape Synthesis and Analysis](https://mlsanity.com/paper/2012.13522/) (2020)
- [Learning Energy-based Spatial-Temporal Generative ConvNets for Dynamic Patterns](https://mlsanity.com/paper/1909.11975/) (2019)
- [Energy-based models for atomic-resolution protein conformations](https://mlsanity.com/paper/2004.13167/) (2020)
- [Wasserstein Introspective Neural Networks](https://mlsanity.com/paper/1711.08875/) (2017)
- [Learning Dynamic Generator Model by Alternating Back-Propagation Through Time](https://mlsanity.com/paper/1812.10587/) (2018)

## Abstract (arXiv)

We propose in this paper, STANLEY, a STochastic gradient ANisotropic LangEvin dYnamics, for sampling high dimensional data. With the growing efficacy and potential of Energy-Based modeling, also known as non-normalized probabilistic modeling, for modeling a generative process of different natures of high dimensional data observations, we present an end-to-end learning algorithm for Energy-Based models (EBM) with the purpose of improving the quality of the resulting sampled data points. While the unknown normalizing constant of EBMs makes the training procedure intractable, resorting to Markov Chain Monte Carlo (MCMC) is in general a viable option. Realizing what MCMC entails for the EBM training, we propose in this paper, a novel high dimensional sampling method, based on an anisotropic stepsize and a gradient-informed covariance matrix, embedded into a discretized Langevin diffusion. We motivate the necessity for an anisotropic update of the negative samples in the Markov Chain by the nonlinearity of the backbone of the EBM, here a Convolutional Neural Network. Our resulting method, namely STANLEY, is an optimization algorithm for training Energy-Based models via our newly introduced MCMC method. We provide a theoretical understanding of our sampling scheme by proving that the sampler leads to a geometrically uniformly ergodic Markov Chain. Several image generation experiments are provided in our paper to show the effectiveness of our method.