# A Hebbian/Anti-Hebbian Network Derived from Online Non-Negative Matrix Factorization Can Cluster and Discover Sparse Features

arXiv: https://arxiv.org/abs/1503.00680 · HTML: https://mlsanity.com/paper/1503.00680/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Mar 2015.

## Built on by (most cited first, 10 of 10)

- [SoftHebb: Bayesian Inference in Unsupervised Hebbian Soft Winner-Take-All Networks](https://mlsanity.com/paper/2107.05747/) (2021)
- [A Normative and Biologically Plausible Algorithm for Independent Component Analysis](https://mlsanity.com/paper/2111.08858/) (2021)
- [Biologically-Plausible Determinant Maximization Neural Networks for Blind Separation of Correlated Sources](https://mlsanity.com/paper/2209.12894/) (2022)
- [Normative framework for deriving neural networks with multi-compartmental neurons and non-Hebbian plasticity](https://mlsanity.com/paper/2302.10051/) (2023)
- [A Similarity-preserving Neural Network Trained on Transformed Images Recapitulates Salient Features of the Fly Motion Detection Circuit](https://mlsanity.com/paper/2102.05503/) (2021)
- [Duality Principle and Biologically Plausible Learning: Connecting the Representer Theorem and Hebbian Learning](https://mlsanity.com/paper/2309.16687/) (2023)
- [Revisiting Gaussian Neurons for Online Clustering with Unknown Number of Clusters](https://mlsanity.com/paper/2205.00920/) (2022)
- [Kernel similarity matching with Hebbian neural networks](https://mlsanity.com/paper/2204.07475/) (2022)
- [Similarity Matching Networks: Hebbian Learning and Convergence Over Multiple Time Scales](https://mlsanity.com/paper/2506.06134/) (2025)
- [How Optimality Structures Sparse Dictionaries: Theory for Interpreting SAE Representations](https://mlsanity.com/paper/2606.02385/) (2026)

## Abstract (arXiv)

Despite our extensive knowledge of biophysical properties of neurons, there is no commonly accepted algorithmic theory of neuronal function. Here we explore the hypothesis that single-layer neuronal networks perform online symmetric nonnegative matrix factorization (SNMF) of the similarity matrix of the streamed data. By starting with the SNMF cost function we derive an online algorithm, which can be implemented by a biologically plausible network with local learning rules. We demonstrate that such network performs soft clustering of the data as well as sparse feature discovery. The derived algorithm replicates many known aspects of sensory anatomy and biophysical properties of neurons including unipolar nature of neuronal activity and synaptic weights, local synaptic plasticity rules and the dependence of learning rate on cumulative neuronal activity. Thus, we make a step towards an algorithmic theory of neuronal function, which should facilitate large-scale neural circuit simulations and biologically inspired artificial intelligence.