# μPC: Scaling Predictive Coding to 100+ Layer Networks

arXiv: https://arxiv.org/abs/2505.13124 · HTML: https://mlsanity.com/paper/2505.13124/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 May 2025.

## Built on by (most cited first, 15 of 15)

- [Towards the Training of Deeper Predictive Coding Neural Networks](https://mlsanity.com/paper/2506.23800/) (2025)
- [ePC: Fast and Deep Predictive Coding in Digital Simulation](https://mlsanity.com/paper/2505.20137/) (2025)
- [Faster Predictive Coding Networks via Better Initialization](https://mlsanity.com/paper/2601.20895/) (2026)
- [Training a Predictive Coding Network on ImageNet using Equilibrium Propagation](https://mlsanity.com/paper/2606.03584/) (2026)
- [On the Infinite Width and Depth Limits of Predictive Coding Networks](https://mlsanity.com/paper/2602.07697/) (2026)
- [Efficient Continual Learning in Language Models via Thalamically Routed Cortical Columns](https://mlsanity.com/paper/2602.22479/) (2026)
- [Cross-Entropy Is Load-Bearing: A Pre-Registered Scope Test of the K-Way Energy Probe on Bidirectional Predictive Coding](https://mlsanity.com/paper/2604.21286/) (2026)
- [Closed-form predictive coding via hierarchical Gaussian filters](https://mlsanity.com/paper/2605.20293/) (2026)
- [Augmented Lagrangian Predictive Coding](https://mlsanity.com/paper/2605.31022/) (2026)
- [Local Reinforcement Learning with Action-Conditioned Root Mean Squared Q-Functions](https://mlsanity.com/paper/2510.06649/) (2025)
- [Error Highways: Scaling Predictive Coding to Very Deep Networks](https://mlsanity.com/paper/2606.22744/) (2026)
- [Weight Feedback Computes the Jacobian Transpose Locally in Modern Deep Networks](https://mlsanity.com/paper/2607.13380/) (2026)
- [A Study of Hidden-State Optimization Order in Predictive Coding Networks](https://mlsanity.com/paper/2609.00686/) (2026)
- [Analog-Friendly Predictive Coding without Activation Derivatives](https://mlsanity.com/paper/2609.32350/) (2026)
- [DeltaNet-PC: Test-Time Memory Without Backpropagation Through Time](https://mlsanity.com/paper/2610.15185/) (2026)

## Abstract (arXiv)

The biological implausibility of backpropagation (BP) has motivated many alternative, brain-inspired algorithms that attempt to rely only on local information, such as predictive coding (PC) and equilibrium propagation. However, these algorithms have notoriously struggled to train very deep networks, preventing them from competing with BP in large-scale settings. Indeed, scaling PC networks (PCNs) has recently been posed as a challenge for the community (Pinchetti et al., 2024). Here, we show that 100+ layer PCNs can be trained reliably using a Depth-μP parameterisation (Yang et al., 2023; Bordelon et al., 2023) which we call "μPC". By analysing the scaling behaviour of PCNs, we reveal several pathologies that make standard PCNs difficult to train at large depths. We then show that, despite addressing only some of these instabilities, μPC allows stable training of very deep (up to 128-layer) residual networks on simple classification tasks with competitive performance and little tuning compared to current benchmarks. Moreover, μPC enables zero-shot transfer of both weight and activity learning rates across widths and depths. Our results serve as a first step towards scaling PC to more complex architectures and have implications for other local algorithms. Code for μPC is made available as part of a JAX library for PCNs.