# Predictive Coding Networks and Inference Learning: Tutorial and Survey

arXiv: https://arxiv.org/abs/2407.04117 · HTML: https://mlsanity.com/paper/2407.04117/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 70 of its 146 references that are in the map.
- First posted on arXiv 4 Jul 2024.

## Builds on (most cited first, 30 of 70)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Wide Residual Networks](https://mlsanity.com/paper/1605.07146/) (2016)
- [Variational Inference with Normalizing Flows](https://mlsanity.com/paper/1505.05770/) (2015)
- [Equilibrium Propagation: Bridging the Gap Between Energy-Based Models and Backpropagation](https://mlsanity.com/paper/1602.05179/) (2016)
- [Dense Associative Memory for Pattern Recognition](https://mlsanity.com/paper/1606.01164/) (2016)
- [Deep Predictive Coding Networks for Video Prediction and Unsupervised Learning](https://mlsanity.com/paper/1605.08104/) (2016)
- [Deep Information Propagation](https://mlsanity.com/paper/1611.01232/) (2016)
- [The Principles of Deep Learning Theory](https://mlsanity.com/paper/2106.10165/) (2021)
- [Difference Target Propagation](https://mlsanity.com/paper/1412.7525/) (2014)
- [A high-bias, low-variance introduction to Machine Learning for physicists](https://mlsanity.com/paper/1803.08823/) (2018)
- [Tensor Programs VI: Feature Learning in Infinite-Depth Neural Networks](https://mlsanity.com/paper/2310.02244/) (2023)
- [Predictive Coding Approximates Backprop along Arbitrary Computation Graphs](https://mlsanity.com/paper/2006.04182/) (2020)
- [Toward Next-Generation Artificial Intelligence: Catalyzing the NeuroAI Revolution](https://mlsanity.com/paper/2210.08340/) (2022)
- [Predictive Coding: a Theoretical and Experimental Review](https://mlsanity.com/paper/2107.12979/) (2021)
- [The free energy principle for action and perception: A mathematical review](https://mlsanity.com/paper/1705.09156/) (2017)
- [Depthwise Hyperparameter Transfer in Residual Networks: Dynamics and Scaling Limit](https://mlsanity.com/paper/2309.16620/) (2023)
- [Position: Bayesian Deep Learning is Needed in the Age of Large-Scale AI](https://mlsanity.com/paper/2402.00809/) (2024)
- [Active Inference in Robotics and Artificial Agents: Survey and Challenges](https://mlsanity.com/paper/2112.01871/) (2021)
- [Predictive Coding: Towards a Future of Deep Learning beyond Backpropagation?](https://mlsanity.com/paper/2202.09467/) (2022)
- [The Neural Coding Framework for Learning Generative Models](https://mlsanity.com/paper/2012.03405/) (2020)
- [Associative Memories via Predictive Coding](https://mlsanity.com/paper/2109.08063/) (2021)
- [World Models and Predictive Coding for Cognitive and Developmental Robotics: Frontiers and Challenges](https://mlsanity.com/paper/2301.05832/) (2023)

## Built on by (most cited first, 7 of 7)

- [pyhgf: A neural network library for predictive coding](https://mlsanity.com/paper/2410.09206/) (2024)
- [JPC: Flexible Inference for Predictive Coding Networks in JAX](https://mlsanity.com/paper/2412.03676/) (2024)
- [Predictive Coding Graphs are a Superset of Feedforward Neural Networks](https://mlsanity.com/paper/2603.06142/) (2026)
- [Closed-form predictive coding via hierarchical Gaussian filters](https://mlsanity.com/paper/2605.20293/) (2026)
- [Predictive Coding-based Deep Neural Network Fine-tuning for Computationally Efficient Domain Adaptation](https://mlsanity.com/paper/2509.20269/) (2025)
- [Vision Hopfield Memory Networks for Image Recognition](https://mlsanity.com/paper/2603.25157/) (2026)
- [Single State Update Predictive Coding training for Time Series Forecasting and Anomaly Detection](https://mlsanity.com/paper/2608.24697/) (2026)

## Abstract (arXiv)

Recent years have witnessed a growing call for renewed emphasis on neuroscience-inspired approaches in artificial intelligence research, under the banner of NeuroAI. A prime example of this is predictive coding networks (PCNs), based on the neuroscientific framework of predictive coding. This framework views the brain as a hierarchical Bayesian inference model that minimizes prediction errors through feedback connections. Unlike traditional neural networks trained with backpropagation (BP), PCNs utilize inference learning (IL), a more biologically plausible algorithm that explains patterns of neural activity that BP cannot. Historically, IL has been more computationally intensive, but recent advancements have demonstrated that it can achieve higher efficiency than BP with sufficient parallelization. Furthermore, PCNs can be mathematically considered a superset of traditional feedforward neural networks (FNNs), significantly extending the range of trainable architectures. As inherently probabilistic (graphical) latent variable models, PCNs provide a versatile framework for both supervised learning and unsupervised (generative) modeling that goes beyond traditional artificial neural networks. This work provides a comprehensive review and detailed formal specification of PCNs, particularly situating them within the context of modern ML methods. This positions PC as a promising framework for future ML innovations.