# On the relationship between predictive coding and backpropagation

arXiv: https://arxiv.org/abs/2106.13082 · HTML: https://mlsanity.com/paper/2106.13082/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 40 references that are in the map.
- First posted on arXiv 20 Jun 2021.

## Builds on (most cited first, 10 of 10)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Equilibrium Propagation: Bridging the Gap Between Energy-Based Models and Backpropagation](https://mlsanity.com/paper/1602.05179/) (2016)
- [Predictive Coding Approximates Backprop along Arbitrary Computation Graphs](https://mlsanity.com/paper/2006.04182/) (2020)
- [The free energy principle for action and perception: A mathematical review](https://mlsanity.com/paper/1705.09156/) (2017)
- [Two Routes to Scalable Credit Assignment without Weight Symmetry](https://mlsanity.com/paper/2003.01513/) (2020)
- [Credit Assignment Through Broadcasting a Global Error Vector](https://mlsanity.com/paper/2106.04089/) (2021)
- [Predictive Coding Can Do Exact Backpropagation on Convolutional and Recurrent Neural Networks](https://mlsanity.com/paper/2103.03725/) (2021)
- [Cortical credit assignment by Hebbian, neuromodulatory and inhibitory plasticity](https://mlsanity.com/paper/1911.00307/) (2019)

## Built on by (most cited first, 20 of 20)

- [Holomorphic Equilibrium Propagation Computes Exact Gradients Through Finite Size Oscillations](https://mlsanity.com/paper/2209.00530/) (2022)
- [Brain-inspired Computational Intelligence via Predictive Coding](https://mlsanity.com/paper/2308.07870/) (2023)
- [Predictive Coding as a Neuromorphic Alternative to Backpropagation: A Critical Evaluation](https://mlsanity.com/paper/2304.02658/) (2023)
- [Predictive Coding Networks and Inference Learning: Tutorial and Survey](https://mlsanity.com/paper/2407.04117/) (2024)
- [Local Loss Optimization in the Infinite Width: Stable Parameterization of Predictive Coding Networks and Target Propagation](https://mlsanity.com/paper/2411.02001/) (2024)
- [The Compositional Structure of Bayesian Inference](https://mlsanity.com/paper/2305.06112/) (2023)
- [Tight Stability, Convergence, and Robustness Bounds for Predictive Coding Networks](https://mlsanity.com/paper/2410.04708/) (2024)
- [Meta predictive learning model of languages in neural circuits](https://mlsanity.com/paper/2309.04106/) (2023)
- [On the Infinite Width and Depth Limits of Predictive Coding Networks](https://mlsanity.com/paper/2602.07697/) (2026)
- [From seeing to remembering: Images with harder-to-reconstruct representations leave stronger memory traces](https://mlsanity.com/paper/2302.10392/) (2023)
- [Understanding Predictive Coding as an Adaptive Trust-Region Method](https://mlsanity.com/paper/2305.18188/) (2023)
- [Advancing Perception in Artificial Intelligence through Principles of Cognitive Science](https://mlsanity.com/paper/2310.08803/) (2023)
- [Predictive Coding Graphs are a Superset of Feedforward Neural Networks](https://mlsanity.com/paper/2603.06142/) (2026)
- [Augmented Lagrangian Predictive Coding](https://mlsanity.com/paper/2605.31022/) (2026)
- [Differentiable Generalised Predictive Coding](https://mlsanity.com/paper/2112.03378/) (2021)
- [Biologically-Motivated Learning Model for Instructed Visual Processing](https://mlsanity.com/paper/2306.02415/) (2023)
- [Bio-Inspired Artificial Neural Networks based on Predictive Coding](https://mlsanity.com/paper/2508.08762/) (2025)
- [Efficient Online Learning with Predictive Coding Networks: Exploiting Temporal Correlations](https://mlsanity.com/paper/2510.25993/) (2025)
- [Accelerating Hierarchical Sparse Predictive Coding with Hybrid Amortized Inference](https://mlsanity.com/paper/2606.27802/) (2026)
- [Learning in Deep Networks under Dale's Constraint](https://mlsanity.com/paper/2608.06963/) (2026)

## Abstract (arXiv)

Artificial neural networks are often interpreted as abstract models of biological neuronal networks, but they are typically trained using the biologically unrealistic backpropagation algorithm and its variants. Predictive coding has been proposed as a potentially more biologically realistic alternative to backpropagation for training neural networks. This manuscript reviews and extends recent work on the mathematical relationship between predictive coding and backpropagation for training feedforward artificial neural networks on supervised learning tasks. Implications of these results for the interpretation of predictive coding and deep neural networks as models of biological learning are discussed along with a repository of functions, Torch2PC, for performing predictive coding with PyTorch neural network models.