# Sample as You Infer: Predictive Coding With Langevin Dynamics

arXiv: https://arxiv.org/abs/2311.13664 · HTML: https://mlsanity.com/paper/2311.13664/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Nov 2023.

## Built on by (most cited first, 12 of 12)

- [Benchmarking Predictive Coding Networks -- Made Simple](https://mlsanity.com/paper/2407.01163/) (2024)
- [Predictive Coding Networks and Inference Learning: Tutorial and Survey](https://mlsanity.com/paper/2407.04117/) (2024)
- [ePC: Fast and Deep Predictive Coding in Digital Simulation](https://mlsanity.com/paper/2505.20137/) (2025)
- [Bidirectional predictive coding](https://mlsanity.com/paper/2505.23415/) (2025)
- [Divide-and-Conquer Predictive Coding: a structured Bayesian inference algorithm](https://mlsanity.com/paper/2408.05834/) (2024)
- [Faster Predictive Coding Networks via Better Initialization](https://mlsanity.com/paper/2601.20895/) (2026)
- [JPC: Flexible Inference for Predictive Coding Networks in JAX](https://mlsanity.com/paper/2412.03676/) (2024)
- [Bayesian Predictive Coding](https://mlsanity.com/paper/2503.24016/) (2025)
- [Why the Brain Consolidates: Predictive Forgetting for Optimal Generalisation](https://mlsanity.com/paper/2603.04688/) (2026)
- [K-Way Energy Probes for Metacognition Reduce to Softmax in Discriminative Predictive Coding Networks](https://mlsanity.com/paper/2604.11011/) (2026)
- [When in Doubt, Think Slow: Iterative Reasoning with Latent Imagination](https://mlsanity.com/paper/2402.15283/) (2024)
- [PotentialEP: Backprop-Free Training of CIFAR-Scale Generative Models via Equilibrium Propagation](https://mlsanity.com/paper/2610.14409/) (2026)

## Abstract (arXiv)

We present a novel algorithm for parameter learning in generic deep generative models that builds upon the predictive coding (PC) framework of computational neuroscience. Our approach modifies the standard PC algorithm to bring performance on-par and exceeding that obtained from standard variational auto-encoder (VAE) training. By injecting Gaussian noise into the PC inference procedure we re-envision it as an overdamped Langevin sampling, which facilitates optimisation with respect to a tight evidence lower bound (ELBO). We improve the resultant encoder-free training method by incorporating an encoder network to provide an amortised warm-start to our Langevin sampling and test three different objectives for doing so. Finally, to increase robustness to the sampling step size and reduce sensitivity to curvature, we validate a lightweight and easily computable form of preconditioning, inspired by Riemann Manifold Langevin and adaptive optimizers from the SGD literature. We compare against VAEs by training like-for-like generative models using our technique against those trained with standard reparameterisation-trick-based ELBOs. We observe our method out-performs or matches performance across a number of metrics, including sample quality, while converging in a fraction of the number of SGD training iterations.