# Predictive Coding, Variational Autoencoders, and Biological Connections

arXiv: https://arxiv.org/abs/2011.07464 · HTML: https://mlsanity.com/paper/2011.07464/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Nov 2020.

## Built on by (most cited first, 21 of 21)

- [Diffusion Probabilistic Modeling for Video Generation](https://mlsanity.com/paper/2203.09481/) (2022)
- [Predictive Coding: a Theoretical and Experimental Review](https://mlsanity.com/paper/2107.12979/) (2021)
- [Brain-inspired Computational Intelligence via Predictive Coding](https://mlsanity.com/paper/2308.07870/) (2023)
- [A Theoretical Framework for Inference Learning](https://mlsanity.com/paper/2206.00164/) (2022)
- [Poisson Variational Autoencoder](https://mlsanity.com/paper/2405.14473/) (2024)
- [Supervised structure learning](https://mlsanity.com/paper/2311.10300/) (2023)
- [Predictive Coding Networks and Inference Learning: Tutorial and Survey](https://mlsanity.com/paper/2407.04117/) (2024)
- [The Expressive Leaky Memory Neuron: an Efficient and Expressive Phenomenological Neuron Model Can Solve Long-Horizon Tasks](https://mlsanity.com/paper/2306.16922/) (2023)
- [Active Sensing with Predictive Coding and Uncertainty Minimization](https://mlsanity.com/paper/2307.00668/) (2023)
- [Language Model-Based Paired Variational Autoencoders for Robotic Language Learning](https://mlsanity.com/paper/2201.06317/) (2022)
- [Predictive World Models from Real-World Partial Observations](https://mlsanity.com/paper/2301.04783/) (2023)
- [Latent Noise Segmentation: How Neural Noise Leads to the Emergence of Segmentation and Grouping](https://mlsanity.com/paper/2309.16515/) (2023)
- [PredProp: Bidirectional Stochastic Optimization with Precision Weighted Predictive Coding](https://mlsanity.com/paper/2111.08792/) (2021)
- [Learning to Predict Navigational Patterns from Partial Observations](https://mlsanity.com/paper/2304.13242/) (2023)
- [Top-down inference in an early visual cortex inspired hierarchical Variational Autoencoder](https://mlsanity.com/paper/2206.00436/) (2022)
- [Learning beyond sensations: how dreams organize neuronal representations](https://mlsanity.com/paper/2308.01830/) (2023)
- [Negative Binomial Variational Autoencoders for Overdispersed Latent Modeling](https://mlsanity.com/paper/2508.05423/) (2025)
- [Predictive Coding Graphs are a Superset of Feedforward Neural Networks](https://mlsanity.com/paper/2603.06142/) (2026)
- [Differentiable Generalised Predictive Coding](https://mlsanity.com/paper/2112.03378/) (2021)
- [Sensory robustness through top-down feedback and neural stochasticity in recurrent vision models](https://mlsanity.com/paper/2508.07115/) (2025)
- [Predictive Coding with Bayesian Priors via Proximal Gradients](https://mlsanity.com/paper/2606.08374/) (2026)

## Abstract (arXiv)

This paper reviews predictive coding, from theoretical neuroscience, and variational autoencoders, from machine learning, identifying the common origin and mathematical framework underlying both areas. As each area is prominent within its respective field, more firmly connecting these areas could prove useful in the dialogue between neuroscience and machine learning. After reviewing each area, we discuss two possible correspondences implied by this perspective: cortical pyramidal dendrites as analogous to (non-linear) deep networks and lateral inhibition as analogous to normalizing flows. These connections may provide new directions for further investigations in each field.