# Similarity Matching Networks: Hebbian Learning and Convergence Over Multiple Time Scales

arXiv: https://arxiv.org/abs/2506.06134 · HTML: https://mlsanity.com/paper/2506.06134/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 15 of its 42 references that are in the map.
- First posted on arXiv 6 Jun 2025.

## Builds on (most cited first, 15 of 15)

- [Equilibrium Propagation: Bridging the Gap Between Energy-Based Models and Backpropagation](https://mlsanity.com/paper/1602.05179/) (2016)
- [A Hebbian/Anti-Hebbian Neural Network for Linear Subspace Learning: A Derivation from Multidimensional Scaling of Streaming Data](https://mlsanity.com/paper/1503.00669/) (2015)
- [Why do similarity matching objectives lead to Hebbian/anti-Hebbian networks?](https://mlsanity.com/paper/1703.07914/) (2017)
- [Neuroscience-inspired online unsupervised learning algorithms](https://mlsanity.com/paper/1908.01867/) (2019)
- [Beyond Convexity -- Contraction and Global Convergence of Gradient Descent](https://mlsanity.com/paper/1806.06655/) (2018)
- [A Normative Theory of Adaptive Dimensionality Reduction in Neural Networks](https://mlsanity.com/paper/1511.09426/) (2015)
- [Blind nonnegative source separation using biological neural networks](https://mlsanity.com/paper/1706.00382/) (2017)
- [A Hebbian/Anti-Hebbian Network Derived from Online Non-Negative Matrix Factorization Can Cluster and Discover Sparse Features](https://mlsanity.com/paper/1503.00680/) (2015)
- [Biologically plausible single-layer networks for nonnegative independent component analysis](https://mlsanity.com/paper/2010.12632/) (2020)
- [A Hebbian/Anti-Hebbian Network for Online Sparse Dictionary Learning Derived from Symmetric Matrix Factorization](https://mlsanity.com/paper/1503.00690/) (2015)
- [Modeling and Contractivity of Neural-Synaptic Networks with Hebbian Learning](https://mlsanity.com/paper/2204.05382/) (2022)
- [Theory of coupled neuronal-synaptic dynamics](https://mlsanity.com/paper/2302.08985/) (2023)
- [Positive Competitive Networks for Sparse Reconstruction](https://mlsanity.com/paper/2311.03821/) (2023)
- [Online Feedback Optimization and Singular Perturbation via Contraction Theory](https://mlsanity.com/paper/2310.07966/) (2023)
- [Scalability in nonlinear network systems affected by delays and disturbances](https://mlsanity.com/paper/2006.07422/) (2020)

## Abstract (arXiv)

A recent breakthrough in biologically-plausible normative frameworks for dimensionality reduction is based upon the similarity matching cost function and the low-rank matrix approximation problem. Despite clear biological interpretation, successful application in several domains, and experimental validation, a formal complete convergence analysis remains elusive. Building on this framework, we consider and analyze a continuous-time neural network, the similarity matching network, for principal subspace projection. Derived from a min-max-min objective, this biologically-plausible network consists of three coupled dynamics evolving at different time scales: neural dynamics, lateral synaptic dynamics, and feedforward synaptic dynamics at the fast, intermediate, and slow time scales, respectively. The feedforward and lateral synaptic dynamics consist of Hebbian and anti-Hebbian learning rules, respectively. By leveraging a multilevel optimization framework, we prove convergence of the dynamics in the offline setting. Specifically, at the first level (fast time scale), we show strong convexity of the cost function and global exponential convergence of the corresponding gradient-flow dynamics. At the second level (intermediate time scale), we prove strong concavity of the cost function and exponential convergence of the corresponding gradient-flow dynamics within the space of positive definite matrices. At the third and final level (slow time scale), we study a non-convex and non-smooth cost function, provide explicit expressions for its global minima, and prove almost sure convergence of the corresponding gradient-flow dynamics to the global minima. These results rely on two empirically motivated conjectures that are supported by thorough numerical experiments. Finally, we validate the effectiveness of our approach via a numerical example.