# Relaxing the Constraints on Predictive Coding Models

arXiv: https://arxiv.org/abs/2010.01047 · HTML: https://mlsanity.com/paper/2010.01047/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Oct 2020.

## Built on by (most cited first, 12 of 12)

- [Predictive Coding: a Theoretical and Experimental Review](https://mlsanity.com/paper/2107.12979/) (2021)
- [Predictive Coding: Towards a Future of Deep Learning beyond Backpropagation?](https://mlsanity.com/paper/2202.09467/) (2022)
- [Learning on Arbitrary Graph Topologies via Predictive Coding](https://mlsanity.com/paper/2201.13180/) (2022)
- [Hybrid Predictive Coding: Inferring, Fast and Slow](https://mlsanity.com/paper/2204.02169/) (2022)
- [Backpropagation at the Infinitesimal Inference Limit of Energy-Based Models: Unifying Predictive Coding, Equilibrium Propagation, and Contrastive Hebbian Learning](https://mlsanity.com/paper/2206.02629/) (2022)
- [A Stable, Fast, and Fully Automatic Learning Algorithm for Predictive Coding Networks](https://mlsanity.com/paper/2212.00720/) (2022)
- [A Theoretical Framework for Inference and Learning in Predictive Coding Networks](https://mlsanity.com/paper/2207.12316/) (2022)
- [Applications of the Free Energy Principle to Machine Learning and Neuroscience](https://mlsanity.com/paper/2107.00140/) (2021)
- [Predictive Coding as a Neuromorphic Alternative to Backpropagation: A Critical Evaluation](https://mlsanity.com/paper/2304.02658/) (2023)
- [Neural Kalman Filtering](https://mlsanity.com/paper/2102.10021/) (2021)
- [Deep Predictive Coding with Bi-directional Propagation for Classification and Reconstruction](https://mlsanity.com/paper/2305.18472/) (2023)
- [Analog-Friendly Predictive Coding without Activation Derivatives](https://mlsanity.com/paper/2609.32350/) (2026)

## Abstract (arXiv)

Predictive coding is an influential theory of cortical function which posits that the principal computation the brain performs, which underlies both perception and learning, is the minimization of prediction errors. While motivated by high-level notions of variational inference, detailed neurophysiological models of cortical microcircuits which can implements its computations have been developed. Moreover, under certain conditions, predictive coding has been shown to approximate the backpropagation of error algorithm, and thus provides a relatively biologically plausible credit-assignment mechanism for training deep networks. However, standard implementations of the algorithm still involve potentially neurally implausible features such as identical forward and backward weights, backward nonlinear derivatives, and 1-1 error unit connectivity. In this paper, we show that these features are not integral to the algorithm and can be removed either directly or through learning additional sets of parameters with Hebbian update rules without noticeable harm to learning performance. Our work thus relaxes current constraints on potential microcircuit designs and hopefully opens up new regions of the design-space for neuromorphic implementations of predictive coding.