# Training End-to-End Analog Neural Networks with Equilibrium Propagation

arXiv: https://arxiv.org/abs/2006.01981 · HTML: https://mlsanity.com/paper/2006.01981/

## Key facts

- Cited by 63 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Jun 2020.

## Built on by (most cited first, 30 of 63)

- [The Forward-Forward Algorithm: Some Preliminary Investigations](https://mlsanity.com/paper/2212.13345/) (2022)
- [Training of Physical Neural Networks](https://mlsanity.com/paper/2406.03372/) (2024)
- [Holomorphic Equilibrium Propagation Computes Exact Gradients Through Finite Size Oscillations](https://mlsanity.com/paper/2209.00530/) (2022)
- [Energy-based learning algorithms for analog computing: a comparative study](https://mlsanity.com/paper/2312.15103/) (2023)
- [Learning on Arbitrary Graph Topologies via Predictive Coding](https://mlsanity.com/paper/2201.13180/) (2022)
- [Brain-inspired Computational Intelligence via Predictive Coding](https://mlsanity.com/paper/2308.07870/) (2023)
- [Training an Ising Machine with Equilibrium Propagation](https://mlsanity.com/paper/2305.18321/) (2023)
- [Beyond backpropagation: bilevel optimization through implicit differentiation and equilibrium propagation](https://mlsanity.com/paper/2205.03076/) (2022)
- [Backpropagation at the Infinitesimal Inference Limit of Energy-Based Models: Unifying Predictive Coding, Equilibrium Propagation, and Contrastive Hebbian Learning](https://mlsanity.com/paper/2206.02629/) (2022)
- [A deep learning theory for neural networks grounded in physics](https://mlsanity.com/paper/2103.09985/) (2021)
- [A Stable, Fast, and Fully Automatic Learning Algorithm for Predictive Coding Networks](https://mlsanity.com/paper/2212.00720/) (2022)
- [Machine Learning Without a Processor: Emergent Learning in a Nonlinear Electronic Metamaterial](https://mlsanity.com/paper/2311.00537/) (2023)
- [Self-Contrastive Forward-Forward Algorithm](https://mlsanity.com/paper/2409.11593/) (2024)
- [Predictive Coding beyond Gaussian Distributions](https://mlsanity.com/paper/2211.03481/) (2022)
- [Agnostic Physics-Driven Deep Learning](https://mlsanity.com/paper/2205.15021/) (2022)
- [Brain-Inspired Machine Intelligence: A Survey of Neurobiologically-Plausible Credit Assignment](https://mlsanity.com/paper/2312.09257/) (2023)
- [Dual Propagation: Accelerating Contrastive Hebbian Learning with Dyadic Neurons](https://mlsanity.com/paper/2302.01228/) (2023)
- [Training Dynamical Binary Neural Networks with Equilibrium Propagation](https://mlsanity.com/paper/2103.08953/) (2021)
- [Forward Learning with Top-Down Feedback: Empirical and Analytical Characterization](https://mlsanity.com/paper/2302.05440/) (2023)
- [Learning by non-interfering feedback chemical signaling in physical networks](https://mlsanity.com/paper/2203.12098/) (2022)
- [Frequency propagation: Multi-mechanism learning in nonlinear physical networks](https://mlsanity.com/paper/2208.08862/) (2022)
- [Convolutional Neural Generative Coding: Scaling Predictive Coding to Natural Images](https://mlsanity.com/paper/2211.12047/) (2022)
- [Quantum Equilibrium Propagation: Gradient-Descent Training of Quantum Systems](https://mlsanity.com/paper/2406.00879/) (2024)
- [Improving equilibrium propagation without weight symmetry through Jacobian homeostasis](https://mlsanity.com/paper/2309.02214/) (2023)
- [Quantum Equilibrium Propagation for efficient training of quantum systems based on Onsager reciprocity](https://mlsanity.com/paper/2406.06482/) (2024)
- [Towards the Training of Deeper Predictive Coding Neural Networks](https://mlsanity.com/paper/2506.23800/) (2025)
- [A Gradient Estimator for Time-Varying Electrical Networks with Non-Linear Dissipation](https://mlsanity.com/paper/2103.05636/) (2021)
- [A Fast Algorithm to Simulate Nonlinear Resistive Networks](https://mlsanity.com/paper/2402.11674/) (2024)
- [Fixed points of nonnegative neural networks](https://mlsanity.com/paper/2106.16239/) (2021)
- [Equilibrium Aggregation: Encoding Sets via Optimization](https://mlsanity.com/paper/2202.12795/) (2022)

## Abstract (arXiv)

We introduce a principled method to train end-to-end analog neural networks by stochastic gradient descent. In these analog neural networks, the weights to be adjusted are implemented by the conductances of programmable resistive devices such as memristors [Chua, 1971], and the nonlinear transfer functions (or `activation functions') are implemented by nonlinear components such as diodes. We show mathematically that a class of analog neural networks (called nonlinear resistive networks) are energy-based models: they possess an energy function as a consequence of Kirchhoff's laws governing electrical circuits. This property enables us to train them using the Equilibrium Propagation framework [Scellier and Bengio, 2017]. Our update rule for each conductance, which is local and relies solely on the voltage drop across the corresponding resistor, is shown to compute the gradient of the loss function. Our numerical simulations, which use the SPICE-based Spectre simulation framework to simulate the dynamics of electrical circuits, demonstrate training on the MNIST classification task, performing comparably or better than equivalent-size software-based neural networks. Our work can guide the development of a new generation of ultra-fast, compact and low-power neural networks supporting on-chip learning.