# Predictive Coding: Towards a Future of Deep Learning beyond Backpropagation?

arXiv: https://arxiv.org/abs/2202.09467 · HTML: https://mlsanity.com/paper/2202.09467/

## Key facts

- Cited by 45 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 75 references that are in the map.
- First posted on arXiv 18 Feb 2022.

## Builds on (most cited first, 24 of 24)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Review](https://mlsanity.com/paper/1805.00909/) (2018)
- [Hopfield Networks is All You Need](https://mlsanity.com/paper/2008.02217/) (2020)
- [Deep Predictive Coding Networks for Video Prediction and Unsupervised Learning](https://mlsanity.com/paper/1605.08104/) (2016)
- [Predictive Coding Approximates Backprop along Arbitrary Computation Graphs](https://mlsanity.com/paper/2006.04182/) (2020)
- [Predictive Coding: a Theoretical and Experimental Review](https://mlsanity.com/paper/2107.12979/) (2021)
- [The free energy principle for action and perception: A mathematical review](https://mlsanity.com/paper/1705.09156/) (2017)
- [Active Inference in Robotics and Artificial Agents: Survey and Challenges](https://mlsanity.com/paper/2112.01871/) (2021)
- [Overparameterized Neural Networks Implement Associative Memory](https://mlsanity.com/paper/1909.12362/) (2019)
- [The Neural Coding Framework for Learning Generative Models](https://mlsanity.com/paper/2012.03405/) (2020)
- [Associative Memories via Predictive Coding](https://mlsanity.com/paper/2109.08063/) (2021)
- [Biologically Motivated Algorithms for Propagating Local Target Representations](https://mlsanity.com/paper/1805.11703/) (2018)
- [On the Relationship Between Active Inference and Control as Inference](https://mlsanity.com/paper/2006.12964/) (2020)
- [Learning on Arbitrary Graph Topologies via Predictive Coding](https://mlsanity.com/paper/2201.13180/) (2022)
- [Deep Predictive Coding Network with Local Recurrent Processing for Object Recognition](https://mlsanity.com/paper/1805.07526/) (2018)
- [End-to-End Pixel-Based Deep Active Inference for Body Perception and Action](https://mlsanity.com/paper/2001.05847/) (2019)
- [Adaptive robot body learning and estimation through predictive coding](https://mlsanity.com/paper/1805.03104/) (2018)
- [A Novel Adaptive Controller for Robot Manipulators based on Active Inference](https://mlsanity.com/paper/1909.12768/) (2019)
- [Relaxing the Constraints on Predictive Coding Models](https://mlsanity.com/paper/2010.01047/) (2020)
- [Habitual and Reflective Control in Hierarchical Predictive Coding](https://mlsanity.com/paper/2109.00866/) (2021)
- [A Brain Inspired Learning Algorithm for the Perception of a Quadrotor in Wind](https://mlsanity.com/paper/2109.11971/) (2021)

## Built on by (most cited first, 30 of 45)

- [Consciousness in Artificial Intelligence: Insights from the Science of Consciousness](https://mlsanity.com/paper/2308.08708/) (2023)
- [Neither hype nor gloom do DNNs justice](https://mlsanity.com/paper/2312.05355/) (2023)
- [Brain-inspired Computational Intelligence via Predictive Coding](https://mlsanity.com/paper/2308.07870/) (2023)
- [Hybrid Predictive Coding: Inferring, Fast and Slow](https://mlsanity.com/paper/2204.02169/) (2022)
- [Backpropagation at the Infinitesimal Inference Limit of Energy-Based Models: Unifying Predictive Coding, Equilibrium Propagation, and Contrastive Hebbian Learning](https://mlsanity.com/paper/2206.02629/) (2022)
- [A Stable, Fast, and Fully Automatic Learning Algorithm for Predictive Coding Networks](https://mlsanity.com/paper/2212.00720/) (2022)
- [A Theoretical Framework for Inference and Learning in Predictive Coding Networks](https://mlsanity.com/paper/2207.12316/) (2022)
- [Predictive Coding beyond Gaussian Distributions](https://mlsanity.com/paper/2211.03481/) (2022)
- [Only Strict Saddles in the Energy Landscape of Predictive Coding Networks?](https://mlsanity.com/paper/2408.11979/) (2024)
- [Predictive Coding as a Neuromorphic Alternative to Backpropagation: A Critical Evaluation](https://mlsanity.com/paper/2304.02658/) (2023)
- [Predictive Coding Networks and Inference Learning: Tutorial and Survey](https://mlsanity.com/paper/2407.04117/) (2024)
- [Block-local learning with probabilistic latent representations](https://mlsanity.com/paper/2305.14974/) (2023)
- [Backpropagation-free Spiking Neural Networks with the Forward-Forward Algorithm](https://mlsanity.com/paper/2502.20411/) (2025)
- [ePC: Fast and Deep Predictive Coding in Digital Simulation](https://mlsanity.com/paper/2505.20137/) (2025)
- [Improving deep learning with prior knowledge and cognitive models: A survey on enhancing explainability, adversarial robustness and zero-shot learning](https://mlsanity.com/paper/2403.07078/) (2024)
- [Deep Active Inference Agents for Delayed and Long-Horizon Environments](https://mlsanity.com/paper/2505.19867/) (2025)
- [The Compositional Structure of Bayesian Inference](https://mlsanity.com/paper/2305.06112/) (2023)
- [Tight Stability, Convergence, and Robustness Bounds for Predictive Coding Networks](https://mlsanity.com/paper/2410.04708/) (2024)
- [Meta predictive learning model of languages in neural circuits](https://mlsanity.com/paper/2309.04106/) (2023)
- [Report on Candidate Computational Indicators for Conscious Valenced Experience](https://mlsanity.com/paper/2404.16696/) (2024)
- [pyhgf: A neural network library for predictive coding](https://mlsanity.com/paper/2410.09206/) (2024)
- [JPC: Flexible Inference for Predictive Coding Networks in JAX](https://mlsanity.com/paper/2412.03676/) (2024)
- [Bayesian Predictive Coding](https://mlsanity.com/paper/2503.24016/) (2025)
- [Self-orthogonalizing attractor neural networks emerging from the free energy principle](https://mlsanity.com/paper/2505.22749/) (2025)
- [Accelerated Predictive Coding Networks via Direct Kolen-Pollack Feedback Alignment](https://mlsanity.com/paper/2602.15571/) (2026)
- [Beyond the limitations of any imaginable mechanism: large language models and psycholinguistics](https://mlsanity.com/paper/2303.00077/) (2023)
- [Understanding Predictive Coding as an Adaptive Trust-Region Method](https://mlsanity.com/paper/2305.18188/) (2023)
- [Deep Predictive Coding with Bi-directional Propagation for Classification and Reconstruction](https://mlsanity.com/paper/2305.18472/) (2023)
- [Advancing Perception in Artificial Intelligence through Principles of Cognitive Science](https://mlsanity.com/paper/2310.08803/) (2023)
- [Active Inference Meeting Energy-Efficient Control of Parallel and Identical Machines](https://mlsanity.com/paper/2406.09322/) (2024)

## Abstract (arXiv)

The backpropagation of error algorithm used to train deep neural networks has been fundamental to the successes of deep learning. However, it requires sequential backward updates and non-local computations, which make it challenging to parallelize at scale and is unlike how learning works in the brain. Neuroscience-inspired learning algorithms, however, such as predictive coding, which utilize local learning, have the potential to overcome these limitations and advance beyond current deep learning technologies. While predictive coding originated in theoretical neuroscience as a model of information processing in the cortex, recent work has developed the idea into a general-purpose algorithm able to train neural networks using only local computations. In this survey, we review works that have contributed to this perspective and demonstrate the close theoretical connections between predictive coding and backpropagation, as well as works that highlight the multiple advantages of using predictive coding models over backpropagation-trained neural networks. Specifically, we show the substantially greater flexibility of predictive coding networks against equivalent deep neural networks, which can function as classifiers, generators, and associative memories simultaneously, and can be defined on arbitrary graph topologies. Finally, we review direct benchmarks of predictive coding networks on machine learning classification tasks, as well as its close connections to control theory and applications in robotics.