# Holomorphic Equilibrium Propagation Computes Exact Gradients Through Finite Size Oscillations

arXiv: https://arxiv.org/abs/2209.00530 · HTML: https://mlsanity.com/paper/2209.00530/

## Key facts

- Cited by 42 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 85 references that are in the map.
- First posted on arXiv 1 Sep 2022.

## Builds on (most cited first, 30 of 35)

- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Sigmoid-Weighted Linear Units for Neural Network Function Approximation in Reinforcement Learning](https://mlsanity.com/paper/1702.03118/) (2017)
- [A Downsampled Variant of ImageNet as an Alternative to the CIFAR datasets](https://mlsanity.com/paper/1707.08819/) (2017)
- [Deep Learning in Spiking Neural Networks](https://mlsanity.com/paper/1804.08150/) (2018)
- [Equilibrium Propagation: Bridging the Gap Between Energy-Based Models and Backpropagation](https://mlsanity.com/paper/1602.05179/) (2016)
- [Direct Feedback Alignment Provides Learning in Deep Neural Networks](https://mlsanity.com/paper/1609.01596/) (2016)
- [SuperSpike: Supervised learning in multi-layer spiking neural networks](https://mlsanity.com/paper/1705.11146/) (2017)
- [Difference Target Propagation](https://mlsanity.com/paper/1412.7525/) (2014)
- [Assessing the Scalability of Biologically-Motivated Deep Learning Algorithms and Architectures](https://mlsanity.com/paper/1807.04587/) (2018)
- [Dendritic cortical microcircuits approximate the backpropagation algorithm](https://mlsanity.com/paper/1810.11393/) (2018)
- [Predictive Coding Approximates Backprop along Arbitrary Computation Graphs](https://mlsanity.com/paper/2006.04182/) (2020)
- [Deep Learning without Weight Transport](https://mlsanity.com/paper/1904.05391/) (2019)
- [A Survey of Neuromorphic Computing and Neural Networks in Hardware](https://mlsanity.com/paper/1705.06963/) (2017)
- [How Auto-Encoders Could Provide Credit Assignment in Deep Networks via Target Propagation](https://mlsanity.com/paper/1407.7906/) (2014)
- [Scaling Equilibrium Propagation to Deep ConvNets by Drastically Reducing its Gradient Estimator Bias](https://mlsanity.com/paper/2006.03824/) (2020)
- [Training End-to-End Analog Neural Networks with Equilibrium Propagation](https://mlsanity.com/paper/2006.01981/) (2020)
- [A Theoretical Framework for Target Propagation](https://mlsanity.com/paper/2006.14331/) (2020)
- [Direct Feedback Alignment Scales to Modern Deep Learning Tasks and Architectures](https://mlsanity.com/paper/2006.12878/) (2020)
- [Towards Scaling Difference Target Propagation by Learning Backprop Targets](https://mlsanity.com/paper/2201.13415/) (2022)
- [Updates of Equilibrium Prop Match Gradients of Backprop Through Time in an RNN with Static Input](https://mlsanity.com/paper/1905.13633/) (2019)
- [EqSpike: Spike-driven Equilibrium Propagation for Neuromorphic Implementations](https://mlsanity.com/paper/2010.07859/) (2020)
- [Refining activation downsampling with SoftPool](https://mlsanity.com/paper/2101.00440/) (2021)
- [Generalization of Equilibrium Propagation to Vector Field Dynamics](https://mlsanity.com/paper/1808.04873/) (2018)
- [Robust computation with rhythmic spike patterns](https://mlsanity.com/paper/1901.07718/) (2019)
- [On the relationship between predictive coding and backpropagation](https://mlsanity.com/paper/2106.13082/) (2021)
- [Error Forward-Propagation: Reusing Feedforward Connections to Propagate Errors in Deep Learning](https://mlsanity.com/paper/1808.03357/) (2018)
- [Deep learning with asymmetric connections and Hebbian updates](https://mlsanity.com/paper/1812.07965/) (2018)
- [GAIT-prop: A biologically plausible learning rule derived from backpropagation of error](https://mlsanity.com/paper/2006.06438/) (2020)
- [Deriving Differential Target Propagation from Iterating Approximate Inverses](https://mlsanity.com/paper/2007.15139/) (2020)

## Built on by (most cited first, 30 of 42)

- [Training of Physical Neural Networks](https://mlsanity.com/paper/2406.03372/) (2024)
- [Artificial Kuramoto Oscillatory Neurons](https://mlsanity.com/paper/2410.13821/) (2024)
- [Energy-based learning algorithms for analog computing: a comparative study](https://mlsanity.com/paper/2312.15103/) (2023)
- [Brain-inspired Computational Intelligence via Predictive Coding](https://mlsanity.com/paper/2308.07870/) (2023)
- [Benchmarking Predictive Coding Networks -- Made Simple](https://mlsanity.com/paper/2407.01163/) (2024)
- [Training an Ising Machine with Equilibrium Propagation](https://mlsanity.com/paper/2305.18321/) (2023)
- [Online learning of long-range dependencies](https://mlsanity.com/paper/2305.15947/) (2023)
- [How connectivity structure shapes rich and lazy learning in neural circuits](https://mlsanity.com/paper/2310.08513/) (2023)
- [Dual Propagation: Accelerating Contrastive Hebbian Learning with Dyadic Neurons](https://mlsanity.com/paper/2302.01228/) (2023)
- [Equilibrium Propagation: the Quantum and the Thermal Cases](https://mlsanity.com/paper/2405.08467/) (2024)
- [Quantum Equilibrium Propagation: Gradient-Descent Training of Quantum Systems](https://mlsanity.com/paper/2406.00879/) (2024)
- [Improving equilibrium propagation without weight symmetry through Jacobian homeostasis](https://mlsanity.com/paper/2309.02214/) (2023)
- [Distance-Forward Learning: Enhancing the Forward-Forward Algorithm Towards High-Performance On-Chip Learning](https://mlsanity.com/paper/2408.14925/) (2024)
- [Learning long range dependencies through time reversal symmetry breaking](https://mlsanity.com/paper/2506.05259/) (2025)
- [Towards the Training of Deeper Predictive Coding Neural Networks](https://mlsanity.com/paper/2506.23800/) (2025)
- [A Review of Neuroscience-Inspired Machine Learning](https://mlsanity.com/paper/2403.18929/) (2024)
- [Equilibrium Propagation for Learning in Lagrangian Dynamical Systems](https://mlsanity.com/paper/2505.07363/) (2025)
- [Toward Lifelong Learning in Equilibrium Propagation: Sleep-like and Awake Rehearsal for Enhanced Stability](https://mlsanity.com/paper/2508.14081/) (2025)
- [Flexible Phase Dynamics for Bio-Plausible Contrastive Learning](https://mlsanity.com/paper/2302.12431/) (2023)
- [Towards training digitally-tied analog blocks via hybrid gradient computation](https://mlsanity.com/paper/2409.03306/) (2024)
- [Correlative Information Maximization: A Biologically Plausible Approach to Supervised Deep Neural Networks without Weight Symmetry](https://mlsanity.com/paper/2306.04810/) (2023)
- [Equilibrium Propagation for Non-Conservative Systems](https://mlsanity.com/paper/2602.03670/) (2026)
- [Training a Predictive Coding Network on ImageNet using Equilibrium Propagation](https://mlsanity.com/paper/2606.03584/) (2026)
- [Two Tales of Single-Phase Contrastive Hebbian Learning](https://mlsanity.com/paper/2402.08573/) (2024)
- [Noise-based Local Learning using Stochastic Magnetic Tunnel Junctions](https://mlsanity.com/paper/2412.12783/) (2024)
- [Training a Hopfield Variational Autoencoder with Equilibrium Propagation](https://mlsanity.com/paper/2311.15047/) (2023)
- [Unsupervised End-to-End Training with a Self-Defined Target](https://mlsanity.com/paper/2403.12116/) (2024)
- [Can Biologically Plausible Temporal Credit Assignment Rules Match BPTT for Neural Similarity? E-prop as an Example](https://mlsanity.com/paper/2506.06904/) (2025)
- [Scaling Equilibrium Propagation to Deeper Neural Network Architectures](https://mlsanity.com/paper/2509.26003/) (2025)
- [Heterogeneous Time Constants Improve Stability in Equilibrium Propagation](https://mlsanity.com/paper/2603.03402/) (2026)

## Abstract (arXiv)

Equilibrium propagation (EP) is an alternative to backpropagation (BP) that allows the training of deep neural networks with local learning rules. It thus provides a compelling framework for training neuromorphic systems and understanding learning in neurobiology. However, EP requires infinitesimal teaching signals, thereby limiting its applicability in noisy physical systems. Moreover, the algorithm requires separate temporal phases and has not been applied to large-scale problems. Here we address these issues by extending EP to holomorphic networks. We show analytically that this extension naturally leads to exact gradients even for finite-amplitude teaching signals. Importantly, the gradient can be computed as the first Fourier coefficient from finite neuronal activity oscillations in continuous time without requiring separate phases. Further, we demonstrate in numerical simulations that our approach permits robust estimation of gradients in the presence of noise and that deeper models benefit from the finite teaching signals. Finally, we establish the first benchmark for EP on the ImageNet 32x32 dataset and show that it matches the performance of an equivalent network trained with BP. Our work provides analytical insights that enable scaling EP to large-scale problems and establishes a formal framework for how oscillations could support learning in biological and neuromorphic systems.