# Spike-timing-dependent Hebbian learning as noisy gradient descent

arXiv: https://arxiv.org/abs/2505.10272 · HTML: https://mlsanity.com/paper/2505.10272/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 56 references that are in the map.
- First posted on arXiv 15 May 2025.

## Builds on (most cited first, 12 of 12)

- [Equilibrium Propagation: Bridging the Gap Between Energy-Based Models and Backpropagation](https://mlsanity.com/paper/1602.05179/) (2016)
- [A Variational Perspective on Accelerated Methods in Optimization](https://mlsanity.com/paper/1603.04245/) (2016)
- [On the Almost Sure Convergence of Stochastic Gradient Descent in Non-Convex Problems](https://mlsanity.com/paper/2006.11144/) (2020)
- [The Information Geometry of Mirror Descent](https://mlsanity.com/paper/1310.7780/) (2013)
- [Hessian Riemannian gradient flows in convex programming](https://mlsanity.com/paper/1811.10331/) (2018)
- [Streaming k-PCA: Efficient guarantees for Oja's algorithm, beyond rank-one updates](https://mlsanity.com/paper/2102.03646/) (2021)
- [Riemannian game dynamics](https://mlsanity.com/paper/1603.09173/) (2016)
- [Convergence rates for the Adam optimizer](https://mlsanity.com/paper/2407.21078/) (2024)
- [Oja's Algorithm for Streaming Sparse PCA](https://mlsanity.com/paper/2402.07240/) (2024)
- [On the choice of metric in gradient-based theories of brain function](https://mlsanity.com/paper/1805.11851/) (2018)
- [Winner-Take-All Computation in Spiking Neural Networks](https://mlsanity.com/paper/1904.12591/) (2019)
- [Spike-Based Winner-Take-All Computation: Fundamental Limits and Order-Optimal Circuits](https://mlsanity.com/paper/1904.10399/) (2019)

## Abstract (arXiv)

Hebbian learning is a key principle underlying learning in biological neural networks. We relate a Hebbian spike-timing-dependent plasticity rule to noisy gradient descent with respect to a non-convex loss function on the probability simplex. Despite the constant injection of noise and the non-convexity of the underlying optimization problem, one can rigorously prove that the considered Hebbian learning dynamic identifies the presynaptic neuron with the highest activity and that the convergence is exponentially fast in the number of iterations. This is non-standard and surprising as typically noisy gradient descent with fixed noise level only converges to a stationary regime where the noise causes the dynamic to fluctuate around a minimiser.