# Energy-based learning algorithms for analog computing: a comparative study

arXiv: https://arxiv.org/abs/2312.15103 · HTML: https://mlsanity.com/paper/2312.15103/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 52 references that are in the map.
- First posted on arXiv 22 Dec 2023.

## Builds on (most cited first, 27 of 27)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Your Classifier is Secretly an Energy Based Model and You Should Treat it Like One](https://mlsanity.com/paper/1912.03263/) (2019)
- [Equilibrium Propagation: Bridging the Gap Between Energy-Based Models and Backpropagation](https://mlsanity.com/paper/1602.05179/) (2016)
- [Learning Non-Convergent Non-Persistent Short-Run MCMC Toward Energy-Based Model](https://mlsanity.com/paper/1904.09770/) (2019)
- [Scaling Equilibrium Propagation to Deep ConvNets by Drastically Reducing its Gradient Estimator Bias](https://mlsanity.com/paper/2006.03824/) (2020)
- [Training End-to-End Analog Neural Networks with Equilibrium Propagation](https://mlsanity.com/paper/2006.01981/) (2020)
- [FIRST: A Framework for Optimizing Information Quality in Mobile Crowdsensing Systems](https://mlsanity.com/paper/1804.11147/) (2018)
- [Holomorphic Equilibrium Propagation Computes Exact Gradients Through Finite Size Oscillations](https://mlsanity.com/paper/2209.00530/) (2022)
- [Updates of Equilibrium Prop Match Gradients of Backprop Through Time in an RNN with Static Input](https://mlsanity.com/paper/1905.13633/) (2019)
- [Training an Ising Machine with Equilibrium Propagation](https://mlsanity.com/paper/2305.18321/) (2023)
- [Beyond backpropagation: bilevel optimization through implicit differentiation and equilibrium propagation](https://mlsanity.com/paper/2205.03076/) (2022)
- [Backpropagation at the Infinitesimal Inference Limit of Energy-Based Models: Unifying Predictive Coding, Equilibrium Propagation, and Contrastive Hebbian Learning](https://mlsanity.com/paper/2206.02629/) (2022)
- [Training Deep Neural Networks via Direct Loss Minimization](https://mlsanity.com/paper/1511.06411/) (2015)
- [A deep learning theory for neural networks grounded in physics](https://mlsanity.com/paper/2103.09985/) (2021)
- [The least-control principle for local learning at equilibrium](https://mlsanity.com/paper/2207.01332/) (2022)
- [Agnostic Physics-Driven Deep Learning](https://mlsanity.com/paper/2205.15021/) (2022)
- [A contrastive rule for meta-learning](https://mlsanity.com/paper/2104.01677/) (2021)
- [Dual Propagation: Accelerating Contrastive Hebbian Learning with Dyadic Neurons](https://mlsanity.com/paper/2302.01228/) (2023)
- [Training Dynamical Binary Neural Networks with Equilibrium Propagation](https://mlsanity.com/paper/2103.08953/) (2021)
- [Learning by non-interfering feedback chemical signaling in physical networks](https://mlsanity.com/paper/2203.12098/) (2022)
- [Frequency propagation: Multi-mechanism learning in nonlinear physical networks](https://mlsanity.com/paper/2208.08862/) (2022)
- [Bounds all around: training energy-based models with bidirectional bounds](https://mlsanity.com/paper/2111.00929/) (2021)
- [A Gradient Estimator for Time-Varying Electrical Networks with Non-Linear Dissipation](https://mlsanity.com/paper/2103.05636/) (2021)
- [Desynchronous Learning in a Physics-Driven Learning Network](https://mlsanity.com/paper/2201.04626/) (2022)
- [Predicting Flat-Fading Channels via Meta-Learned Closed-Form Linear Filters and Equilibrium Propagation](https://mlsanity.com/paper/2110.00414/) (2021)
- [Physical Computing for Materials Acceleration Platforms](https://mlsanity.com/paper/2208.08566/) (2022)

## Built on by (most cited first, 30 of 37)

- [Training of Physical Neural Networks](https://mlsanity.com/paper/2406.03372/) (2024)
- [Benchmarking Predictive Coding Networks -- Made Simple](https://mlsanity.com/paper/2407.01163/) (2024)
- [Self-Contrastive Forward-Forward Algorithm](https://mlsanity.com/paper/2409.11593/) (2024)
- [Quantum Equilibrium Propagation: Gradient-Descent Training of Quantum Systems](https://mlsanity.com/paper/2406.00879/) (2024)
- [Only Strict Saddles in the Energy Landscape of Predictive Coding Networks?](https://mlsanity.com/paper/2408.11979/) (2024)
- [Quantum Equilibrium Propagation for efficient training of quantum systems based on Onsager reciprocity](https://mlsanity.com/paper/2406.06482/) (2024)
- [Distance-Forward Learning: Enhancing the Forward-Forward Algorithm Towards High-Performance On-Chip Learning](https://mlsanity.com/paper/2408.14925/) (2024)
- [Towards Exact Gradient-based Training on Analog In-memory Computing](https://mlsanity.com/paper/2406.12774/) (2024)
- [Learning long range dependencies through time reversal symmetry breaking](https://mlsanity.com/paper/2506.05259/) (2025)
- [Towards the Training of Deeper Predictive Coding Neural Networks](https://mlsanity.com/paper/2506.23800/) (2025)
- [A Review of Neuroscience-Inspired Machine Learning](https://mlsanity.com/paper/2403.18929/) (2024)
- [Equilibrium Propagation for Learning in Lagrangian Dynamical Systems](https://mlsanity.com/paper/2505.07363/) (2025)
- [Analog In-memory Training on General Non-ideal Resistive Elements: The Impact of Response Functions](https://mlsanity.com/paper/2502.06309/) (2025)
- [Towards training digitally-tied analog blocks via hybrid gradient computation](https://mlsanity.com/paper/2409.03306/) (2024)
- [Stochastic Forward-Forward Learning through Representational Dimensionality Compression](https://mlsanity.com/paper/2505.16649/) (2025)
- [Tight Stability, Convergence, and Robustness Bounds for Predictive Coding Networks](https://mlsanity.com/paper/2410.04708/) (2024)
- [Q-Detection: A Quantum-Classical Hybrid Poisoning Attack Detection Method](https://mlsanity.com/paper/2507.06262/) (2025)
- [Faster Predictive Coding Networks via Better Initialization](https://mlsanity.com/paper/2601.20895/) (2026)
- [Training a Predictive Coding Network on ImageNet using Equilibrium Propagation](https://mlsanity.com/paper/2606.03584/) (2026)
- [Two Tales of Single-Phase Contrastive Hebbian Learning](https://mlsanity.com/paper/2402.08573/) (2024)
- [Convergence of energy-based learning in linear resistive networks](https://mlsanity.com/paper/2503.00349/) (2025)
- [Toward Practical Equilibrium Propagation: Brain-inspired Recurrent Neural Network with Feedback Regulation and Residual Connections](https://mlsanity.com/paper/2508.11659/) (2025)
- [Adaptive Heavy-Tailed Stochastic Gradient Descent](https://mlsanity.com/paper/2508.21353/) (2025)
- [Accelerated Predictive Coding Networks via Direct Kolen-Pollack Feedback Alignment](https://mlsanity.com/paper/2602.15571/) (2026)
- [Unsupervised End-to-End Training with a Self-Defined Target](https://mlsanity.com/paper/2403.12116/) (2024)
- [Circuit realization and hardware linearization of monotone operator equilibrium networks](https://mlsanity.com/paper/2509.13793/) (2025)
- [Scaling Equilibrium Propagation to Deeper Neural Network Architectures](https://mlsanity.com/paper/2509.26003/) (2025)
- [Provable Accuracy Bounds for Hybrid Dynamical Optimization and Sampling](https://mlsanity.com/paper/2410.06397/) (2024)
- [Strengthening the Internal Adversarial Robustness in Lifted Neural Networks](https://mlsanity.com/paper/2503.07818/) (2025)
- [Harnessing uncertainty when learning through Equilibrium Propagation in neural networks](https://mlsanity.com/paper/2503.22810/) (2025)

## Abstract (arXiv)

Energy-based learning algorithms have recently gained a surge of interest due to their compatibility with analog (post-digital) hardware. Existing algorithms include contrastive learning (CL), equilibrium propagation (EP) and coupled learning (CpL), all consisting in contrasting two states, and differing in the type of perturbation used to obtain the second state from the first one. However, these algorithms have never been explicitly compared on equal footing with same models and datasets, making it difficult to assess their scalability and decide which one to select in practice. In this work, we carry out a comparison of seven learning algorithms, namely CL and different variants of EP and CpL depending on the signs of the perturbations. Specifically, using these learning algorithms, we train deep convolutional Hopfield networks (DCHNs) on five vision tasks (MNIST, F-MNIST, SVHN, CIFAR-10 and CIFAR-100). We find that, while all algorithms yield comparable performance on MNIST, important differences in performance arise as the difficulty of the task increases. Our key findings reveal that negative perturbations are better than positive ones, and highlight the centered variant of EP (which uses two perturbations of opposite sign) as the best-performing algorithm. We also endorse these findings with theoretical arguments. Additionally, we establish new SOTA results with DCHNs on all five datasets, both in performance and speed. In particular, our DCHN simulations are 13.5 times faster with respect to Laborieux et al. (2021), which we achieve thanks to the use of a novel energy minimisation algorithm based on asynchronous updates, combined with reduced precision (16 bits).