mlsanityData to 7 Oct 2026

On the relationship between predictive coding and backpropagation

arXiv 2106.13082 ↗First posted 20 Jun 2021q-bio.NC cs.LG cs.NEcited by 20

Not on the map. Why?

Abstract (arXiv)

Artificial neural networks are often interpreted as abstract models of biological neuronal networks, but they are typically trained using the biologically unrealistic backpropagation algorithm and its variants. Predictive coding has been proposed as a potentially more biologically realistic alternative to backpropagation for training neural networks. This manuscript reviews and extends recent work on the mathematical relationship between predictive coding and backpropagation for training feedforward artificial neural networks on supervised learning tasks. Implications of these results for the interpretation of predictive coding and deep neural networks as models of biological learning are discussed along with a repository of functions, Torch2PC, for performing predictive coding with PyTorch neural network models.

Key facts. Cited by 20 papers in the map (data through 7 Oct 2026). Builds on 10 of its 40 references that are in the map. First posted on arXiv 20 Jun 2021.

Builds on

10 of 40 references.

References · 10

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  3. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  4. 2016Equilibrium Propagation: Bridging the Gap Between Energy-Based Models and Backpropagationcited by 251
  5. 2020Predictive Coding Approximates Backprop along Arbitrary Computation Graphscited by 89
  6. 2017The free energy principle for action and perception: A mathematical reviewcited by 66
  7. 2020Two Routes to Scalable Credit Assignment without Weight Symmetrycited by 19
  8. 2021Credit Assignment Through Broadcasting a Global Error Vectorcited by 19
  9. 2021Predictive Coding Can Do Exact Backpropagation on Convolutional and Recurrent Neural Networkscited by 14
  10. 2019Cortical credit assignment by Hebbian, neuromodulatory and inhibitory plasticitycited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 1; 2023: 8; 2024: 3; 2025: 2; 2026: 5.

20 papers.

Sort:

  1. 2022Holomorphic Equilibrium Propagation Computes Exact Gradients Through Finite Size Oscillationscited by 40
  2. 2023Brain-inspired Computational Intelligence via Predictive Codingcited by 30
  3. 2023Predictive Coding as a Neuromorphic Alternative to Backpropagation: A Critical Evaluationcited by 8
  4. 2024Predictive Coding Networks and Inference Learning: Tutorial and Surveycited by 7
  5. 2024Local Loss Optimization in the Infinite Width: Stable Parameterization of Predictive Coding Networks and Target Propagationcited by 4
  6. 2023The Compositional Structure of Bayesian Inferencecited by 3
  7. 2024Tight Stability, Convergence, and Robustness Bounds for Predictive Coding Networkscited by 3
  8. 2023Meta predictive learning model of languages in neural circuitscited by 2
  9. 2026On the Infinite Width and Depth Limits of Predictive Coding Networkscited by 2
  10. 2023From seeing to remembering: Images with harder-to-reconstruct representations leave stronger memory tracescited by 1
  11. 2023Understanding Predictive Coding as an Adaptive Trust-Region Methodcited by 1
  12. 2023Advancing Perception in Artificial Intelligence through Principles of Cognitive Sciencecited by 1
8 more
  1. 2026Predictive Coding Graphs are a Superset of Feedforward Neural Networkscited by 1
  2. 2026Augmented Lagrangian Predictive Codingcited by 1
  3. 2021Differentiable Generalised Predictive Codingcited by 0
  4. 2023Biologically-Motivated Learning Model for Instructed Visual Processingcited by 0
  5. 2025Bio-Inspired Artificial Neural Networks based on Predictive Codingcited by 0
  6. 2025Efficient Online Learning with Predictive Coding Networks: Exploiting Temporal Correlationscited by 0
  7. 2026Accelerating Hierarchical Sparse Predictive Coding with Hybrid Amortized Inferencecited by 0
  8. 2026Learning in Deep Networks under Dale's Constraintcited by 0