# Two Tales of Single-Phase Contrastive Hebbian Learning

arXiv: https://arxiv.org/abs/2402.08573 · HTML: https://mlsanity.com/paper/2402.08573/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 49 references that are in the map.
- First posted on arXiv 13 Feb 2024.

## Builds on (most cited first, 25 of 25)

- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Equilibrium Propagation: Bridging the Gap Between Energy-Based Models and Backpropagation](https://mlsanity.com/paper/1602.05179/) (2016)
- [Direct Feedback Alignment Provides Learning in Deep Neural Networks](https://mlsanity.com/paper/1609.01596/) (2016)
- [Difference Target Propagation](https://mlsanity.com/paper/1412.7525/) (2014)
- [Decoupled Neural Interfaces using Synthetic Gradients](https://mlsanity.com/paper/1608.05343/) (2016)
- [Training Neural Networks with Local Error Signals](https://mlsanity.com/paper/1901.06656/) (2019)
- [Dendritic cortical microcircuits approximate the backpropagation algorithm](https://mlsanity.com/paper/1810.11393/) (2018)
- [Deep Learning without Weight Transport](https://mlsanity.com/paper/1904.05391/) (2019)
- [Towards deep learning with segregated dendrites](https://mlsanity.com/paper/1610.00161/) (2016)
- [Scaling Equilibrium Propagation to Deep ConvNets by Drastically Reducing its Gradient Estimator Bias](https://mlsanity.com/paper/2006.03824/) (2020)
- [Hebbian Deep Learning Without Feedback](https://mlsanity.com/paper/2209.11883/) (2022)
- [Random feedback weights support learning in deep neural networks](https://mlsanity.com/paper/1411.0247/) (2014)
- [Holomorphic Equilibrium Propagation Computes Exact Gradients Through Finite Size Oscillations](https://mlsanity.com/paper/2209.00530/) (2022)
- [Energy-based learning algorithms for analog computing: a comparative study](https://mlsanity.com/paper/2312.15103/) (2023)
- [Towards Scaling Difference Target Propagation by Learning Backprop Targets](https://mlsanity.com/paper/2201.13415/) (2022)
- [Brain-inspired Computational Intelligence via Predictive Coding](https://mlsanity.com/paper/2308.07870/) (2023)
- [Beyond Backprop: Online Alternating Minimization with Auxiliary Variables](https://mlsanity.com/paper/1806.09077/) (2018)
- [Lifted Neural Networks](https://mlsanity.com/paper/1805.01532/) (2018)
- [Fenchel Lifted Networks: A Lagrange Relaxation of Neural Network Training](https://mlsanity.com/paper/1811.08039/) (2018)
- [Drawing Inspiration from Biological Dendrites to Empower Artificial Neural Networks](https://mlsanity.com/paper/2106.07490/) (2021)
- [Dual Propagation: Accelerating Contrastive Hebbian Learning with Dyadic Neurons](https://mlsanity.com/paper/2302.01228/) (2023)
- [Contrastive Learning for Lifted Networks](https://mlsanity.com/paper/1905.02507/) (2019)
- [Improving equilibrium propagation without weight symmetry through Jacobian homeostasis](https://mlsanity.com/paper/2309.02214/) (2023)
- [Bilevel Programs Meet Deep Learning: A Unifying View on Inference Learning Methods](https://mlsanity.com/paper/2105.07231/) (2021)
- [Lifted Regression/Reconstruction Networks](https://mlsanity.com/paper/2005.03452/) (2020)

## Built on by (most cited first, 2 of 2)

- [Equilibrium Propagation for Non-Conservative Systems](https://mlsanity.com/paper/2602.03670/) (2026)
- [Strengthening the Internal Adversarial Robustness in Lifted Neural Networks](https://mlsanity.com/paper/2503.07818/) (2025)

## Abstract (arXiv)

The search for ``biologically plausible'' learning algorithms has converged on the idea of representing gradients as activity differences. However, most approaches require a high degree of synchronization (distinct phases during learning) and introduce substantial computational overhead, which raises doubts regarding their biological plausibility as well as their potential utility for neuromorphic computing. Furthermore, they commonly rely on applying infinitesimal perturbations (nudges) to output units, which is impractical in noisy environments. Recently it has been shown that by modelling artificial neurons as dyads with two oppositely nudged compartments, it is possible for a fully local learning algorithm named ``dual propagation'' to bridge the performance gap to backpropagation, without requiring separate learning phases or infinitesimal nudging. However, the algorithm has the drawback that its numerical stability relies on symmetric nudging, which may be restrictive in biological and analog implementations. In this work we first provide a solid foundation for the objective underlying the dual propagation method, which also reveals a surprising connection with adversarial robustness. Second, we demonstrate how dual propagation is related to a particular adjoint state method, which is stable regardless of asymmetric nudging.