Analog-Friendly Predictive Coding without Activation Derivatives
Not on the map. Why?
Abstract (arXiv)
Predictive coding (PC) is a local, energy-based alternative to backpropagation (BP) whose iterative inference dynamics make it attractive for implementation on analog hardware. However, standard nonlinear PC requires evaluating the derivative of the activation function during both inference and learning, which can be difficult to realise physically. Here, we introduce textitactivation-matched Bregman PC, replacing standard squared-error energies with Bregman divergences matched to the activation function. This formulation eliminates activation derivatives and, when combined with inference via mirror descent, yields local inference and learning rules requiring only weighted sums, local prediction errors, state integration, and the activation function. In digital experiments, Bregman PC performs comparably to standard PC and BP on classification and generative tasks, while preserving characteristic learning dynamics of PC and its convergence to BP under stable large-model parameterisations. These results provide a more analog-friendly formulation of nonlinear PC while retaining its key computational properties.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 28 of its 33 references that are in the map. First posted on arXiv 26 Sep 2026.
Builds on
28 of 33 references.
References · 28
- 2019Deep Equilibrium Modelscited by 615
- 2022On Neural Differential Equationscited by 268
- 2016Equilibrium Propagation: Bridging the Gap Between Energy-Based Models and Backpropagationcited by 259
- 2023Power Hungry Processing: Watts Driving the Cost of AI Deployment?cited by 191
- 2020The Hardware Lotterycited by 160
- 2021Deep physical neural networks enabled by a backpropagation algorithm for arbitrary physical systemscited by 128
- 2023Energy Transformercited by 71
- 2017The free energy principle for action and perception: A mathematical reviewcited by 66
- 2024Training of Physical Neural Networkscited by 49
- 2022Learning on Arbitrary Graph Topologies via Predictive Codingcited by 33
- 2024Benchmarking Predictive Coding Networks -- Made Simplecited by 30
- 2021LocoProp: Enhancing BackProp via Local Loss Optimizationcited by 15
16 more
- 2025μPC: Scaling Predictive Coding to 100+ Layer Networkscited by 15
- 2020Relaxing the Constraints on Predictive Coding Modelscited by 12
- 2022Lifted Bregman Training of Neural Networkscited by 10
- 2024Only Strict Saddles in the Energy Landscape of Predictive Coding Networks?cited by 10
- 2025Solving the compute crisis with physics-based ASICscited by 8
- 2025Towards the Training of Deeper Predictive Coding Neural Networkscited by 7
- 2025ePC: Fast and Deep Predictive Coding in Digital Simulationcited by 5
- 2020Optoelectronic Intelligencecited by 4
- 2020Generalised Perceptron Learningcited by 4
- 2024Local Loss Optimization in the Infinite Width: Stable Parameterization of Predictive Coding Networks and Target Propagationcited by 4
- 2026Training a Predictive Coding Network on ImageNet using Equilibrium Propagationcited by 3
- 2026On the Infinite Width and Depth Limits of Predictive Coding Networkscited by 2
- 2026Generative Models on Analog Hardware with Dynamicscited by 2
- 2026A Blueprint for Equilibrium-Based Differentiable Continuous-Variable Thermodynamic Computingcited by 2
- 2026Physical Foundation Models: Fixed hardware implementations of large-scale neural networkscited by 1
- 2026Augmented Lagrangian Predictive Codingcited by 1
Built on by
Not cited yet.