# Meta-Learning for Stochastic Gradient MCMC

arXiv: https://arxiv.org/abs/1806.04522 · HTML: https://mlsanity.com/paper/1806.04522/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Jun 2018.

## Built on by (most cited first, 16 of 16)

- [VeLO: Training Versatile Learned Optimizers by Scaling Up](https://mlsanity.com/paper/2211.09760/) (2022)
- [Is MC Dropout Bayesian?](https://mlsanity.com/paper/2110.04286/) (2021)
- [BayesDAG: Gradient-Based Posterior Inference for Causal Discovery](https://mlsanity.com/paper/2307.13917/) (2023)
- [Practical tradeoffs between memory, compute, and performance in learned optimizers](https://mlsanity.com/paper/2203.11860/) (2022)
- [S2AC: Energy-Based Reinforcement Learning with Stein Soft Actor Critic](https://mlsanity.com/paper/2405.00987/) (2024)
- [Neural Structure Learning with Stochastic Differential Equations](https://mlsanity.com/paper/2311.03309/) (2023)
- [Structured Stochastic Gradient MCMC](https://mlsanity.com/paper/2107.09028/) (2021)
- [LSB: Local Self-Balancing MCMC in Discrete Spaces](https://mlsanity.com/paper/2109.03867/) (2021)
- [Confidence-Aware Learning for Camouflaged Object Detection](https://mlsanity.com/paper/2106.11641/) (2021)
- [The Benefit of the Doubt: Uncertainty Aware Sensing for Edge Computing Platforms](https://mlsanity.com/paper/2102.05956/) (2021)
- [Machine Learning and the Future of Bayesian Computation](https://mlsanity.com/paper/2304.11251/) (2023)
- [Learning to Explore for Stochastic Gradient MCMC](https://mlsanity.com/paper/2408.09140/) (2024)
- [Approximate blocked Gibbs sampling for Bayesian neural networks](https://mlsanity.com/paper/2208.11389/) (2022)
- [Parameter Expanded Stochastic Gradient Markov Chain Monte Carlo](https://mlsanity.com/paper/2503.00699/) (2025)
- [Meta-Learning an Inference Algorithm for Probabilistic Programs](https://mlsanity.com/paper/2103.00737/) (2021)
- [Bayesian Computation in Deep Learning](https://mlsanity.com/paper/2502.18300/) (2025)

## Abstract (arXiv)

Stochastic gradient Markov chain Monte Carlo (SG-MCMC) has become increasingly popular for simulating posterior samples in large-scale Bayesian modeling. However, existing SG-MCMC schemes are not tailored to any specific probabilistic model, even a simple modification of the underlying dynamical system requires significant physical intuition. This paper presents the first meta-learning algorithm that allows automated design for the underlying continuous dynamics of an SG-MCMC sampler. The learned sampler generalizes Hamiltonian dynamics with state-dependent drift and diffusion, enabling fast traversal and efficient exploration of neural network energy landscapes. Experiments validate the proposed approach on both Bayesian fully connected neural network and Bayesian recurrent neural network tasks, showing that the learned sampler out-performs generic, hand-designed SG-MCMC algorithms, and generalizes to different datasets and larger architectures.