# Updates of Equilibrium Prop Match Gradients of Backprop Through Time in an RNN with Static Input

arXiv: https://arxiv.org/abs/1905.13633 · HTML: https://mlsanity.com/paper/1905.13633/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 May 2019.

## Built on by (most cited first, 30 of 33)

- [Holomorphic Equilibrium Propagation Computes Exact Gradients Through Finite Size Oscillations](https://mlsanity.com/paper/2209.00530/) (2022)
- [Energy-based learning algorithms for analog computing: a comparative study](https://mlsanity.com/paper/2312.15103/) (2023)
- [Hierarchical Associative Memory](https://mlsanity.com/paper/2107.06446/) (2021)
- [Training an Ising Machine with Equilibrium Propagation](https://mlsanity.com/paper/2305.18321/) (2023)
- [Agnostic Physics-Driven Deep Learning](https://mlsanity.com/paper/2205.15021/) (2022)
- [Training Dynamical Binary Neural Networks with Equilibrium Propagation](https://mlsanity.com/paper/2103.08953/) (2021)
- [Equilibrium Propagation: the Quantum and the Thermal Cases](https://mlsanity.com/paper/2405.08467/) (2024)
- [Improving equilibrium propagation without weight symmetry through Jacobian homeostasis](https://mlsanity.com/paper/2309.02214/) (2023)
- [Sequence Learning Using Equilibrium Propagation](https://mlsanity.com/paper/2209.09626/) (2022)
- [Toward Lifelong Learning in Equilibrium Propagation: Sleep-like and Awake Rehearsal for Enhanced Stability](https://mlsanity.com/paper/2508.14081/) (2025)
- [Towards training digitally-tied analog blocks via hybrid gradient computation](https://mlsanity.com/paper/2409.03306/) (2024)
- [Biologically-inspired neuronal adaptation improves learning in neural networks](https://mlsanity.com/paper/2204.14008/) (2022)
- [Equilibrium Propagation for Non-Conservative Systems](https://mlsanity.com/paper/2602.03670/) (2026)
- [Efficient and Flexible Neural Network Training through Layer-wise Feedback Propagation](https://mlsanity.com/paper/2308.12053/) (2023)
- [Toward Practical Equilibrium Propagation: Brain-inspired Recurrent Neural Network with Feedback Regulation and Residual Connections](https://mlsanity.com/paper/2508.11659/) (2025)
- [Training a Hopfield Variational Autoencoder with Equilibrium Propagation](https://mlsanity.com/paper/2311.15047/) (2023)
- [Unsupervised End-to-End Training with a Self-Defined Target](https://mlsanity.com/paper/2403.12116/) (2024)
- [Scaling Equilibrium Propagation to Deeper Neural Network Architectures](https://mlsanity.com/paper/2509.26003/) (2025)
- [Heterogeneous Time Constants Improve Stability in Equilibrium Propagation](https://mlsanity.com/paper/2603.03402/) (2026)
- [Scaling Equilibrium Propagation to Deep ConvNets by Drastically Reducing its Gradient Estimator Bias](https://mlsanity.com/paper/2101.05536/) (2021)
- [Front Contribution instead of Back Propagation](https://mlsanity.com/paper/2106.05569/) (2021)
- [Biologically-Motivated Learning Model for Instructed Visual Processing](https://mlsanity.com/paper/2306.02415/) (2023)
- [Harnessing uncertainty when learning through Equilibrium Propagation in neural networks](https://mlsanity.com/paper/2503.22810/) (2025)
- [Scalable Equilibrium Propagation via Intermediate Error Signals for Deep Convolutional CRNNs](https://mlsanity.com/paper/2508.15989/) (2025)
- [Can Local Learning Match Self-Supervised Backpropagation?](https://mlsanity.com/paper/2601.21683/) (2026)
- [VertAX: a differentiable vertex model for learning epithelial tissue mechanics](https://mlsanity.com/paper/2604.06896/) (2026)
- [The Phase Is the Gradient: Equilibrium Propagation for Frequency Learning in Kuramoto Networks](https://mlsanity.com/paper/2604.10272/) (2026)
- [Dendritic Neural Networks with Equilibrium Propagation](https://mlsanity.com/paper/2605.08135/) (2026)
- [Hybridizing Equilibrium Propagation with Ising Machines for Efficient Energy-Based Learning](https://mlsanity.com/paper/2606.09112/) (2026)
- [Optimizing Energy-based Neural Network Training with Coherent Ising Machine](https://mlsanity.com/paper/2606.09117/) (2026)

## Abstract (arXiv)

Equilibrium Propagation (EP) is a biologically inspired learning algorithm for convergent recurrent neural networks, i.e. RNNs that are fed by a static input x and settle to a steady state. Training convergent RNNs consists in adjusting the weights until the steady state of output neurons coincides with a target y. Convergent RNNs can also be trained with the more conventional Backpropagation Through Time (BPTT) algorithm. In its original formulation EP was described in the case of real-time neuronal dynamics, which is computationally costly. In this work, we introduce a discrete-time version of EP with simplified equations and with reduced simulation time, bringing EP closer to practical machine learning tasks. We first prove theoretically, as well as numerically that the neural and weight updates of EP, computed by forward-time dynamics, are step-by-step equal to the ones obtained by BPTT, with gradients computed backward in time. The equality is strict when the transition function of the dynamics derives from a primitive function and the steady state is maintained long enough. We then show for more standard discrete-time neural network dynamics that the same property is approximately respected and we subsequently demonstrate training with EP with equivalent performance to BPTT. In particular, we define the first convolutional architecture trained with EP achieving ~ 1% test error on MNIST, which is the lowest error reported with EP. These results can guide the development of deep neural networks trained with EP.