# Understanding and Improving Optimization in Predictive Coding Networks

arXiv: https://arxiv.org/abs/2305.13562 · HTML: https://mlsanity.com/paper/2305.13562/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 May 2023.

## Built on by (most cited first, 13 of 13)

- [Brain-inspired Computational Intelligence via Predictive Coding](https://mlsanity.com/paper/2308.07870/) (2023)
- [Benchmarking Predictive Coding Networks -- Made Simple](https://mlsanity.com/paper/2407.01163/) (2024)
- [Self-Contrastive Forward-Forward Algorithm](https://mlsanity.com/paper/2409.11593/) (2024)
- [Only Strict Saddles in the Energy Landscape of Predictive Coding Networks?](https://mlsanity.com/paper/2408.11979/) (2024)
- [Predictive Coding Networks and Inference Learning: Tutorial and Survey](https://mlsanity.com/paper/2407.04117/) (2024)
- [ePC: Fast and Deep Predictive Coding in Digital Simulation](https://mlsanity.com/paper/2505.20137/) (2025)
- [Local Loss Optimization in the Infinite Width: Stable Parameterization of Predictive Coding Networks and Target Propagation](https://mlsanity.com/paper/2411.02001/) (2024)
- [Faster Predictive Coding Networks via Better Initialization](https://mlsanity.com/paper/2601.20895/) (2026)
- [On the Infinite Width and Depth Limits of Predictive Coding Networks](https://mlsanity.com/paper/2602.07697/) (2026)
- [Augmented Lagrangian Predictive Coding](https://mlsanity.com/paper/2605.31022/) (2026)
- [Learning Long-Range Dependencies with Temporal Predictive Coding](https://mlsanity.com/paper/2602.18131/) (2026)
- [Single State Update Predictive Coding training for Time Series Forecasting and Anomaly Detection](https://mlsanity.com/paper/2608.24697/) (2026)
- [A Study of Hidden-State Optimization Order in Predictive Coding Networks](https://mlsanity.com/paper/2609.00686/) (2026)

## Abstract (arXiv)

Backpropagation (BP), the standard learning algorithm for artificial neural networks, is often considered biologically implausible. In contrast, the standard learning algorithm for predictive coding (PC) models in neuroscience, known as the inference learning algorithm (IL), is a promising, bio-plausible alternative. However, several challenges and questions hinder IL's application to real-world problems. For example, IL is computationally demanding, and without memory-intensive optimizers like Adam, IL may converge to poor local minima. Moreover, although IL can reduce loss more quickly than BP, the reasons for these speedups or their robustness remains unclear. In this paper, we tackle these challenges by 1) altering the standard implementation of PC circuits to substantially reduce computation, 2) developing a novel optimizer that improves the convergence of IL without increasing memory usage, and 3) establishing theoretical results that help elucidate the conditions under which IL is sensitive to second and higher-order information.