# A Hebbian/Anti-Hebbian Neural Network for Linear Subspace Learning: A Derivation from Multidimensional Scaling of Streaming Data

arXiv: https://arxiv.org/abs/1503.00669 · HTML: https://mlsanity.com/paper/1503.00669/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Mar 2015.

## Built on by (most cited first, 26 of 26)

- [Disentanglement with Biological Constraints: A Theory of Functional Cell Types](https://mlsanity.com/paper/2210.01768/) (2022)
- [Hebbian Semi-Supervised Learning in a Sample Efficiency Setting](https://mlsanity.com/paper/2103.09002/) (2021)
- [Training Convolutional Neural Networks with the Forward-Forward algorithm](https://mlsanity.com/paper/2312.14924/) (2023)
- [Low Tensor Rank Learning of Neural Dynamics](https://mlsanity.com/paper/2308.11567/) (2023)
- [Constrained Parameter Inference as a Principle for Learning](https://mlsanity.com/paper/2203.13203/) (2022)
- [A Normative and Biologically Plausible Algorithm for Independent Component Analysis](https://mlsanity.com/paper/2111.08858/) (2021)
- [Hebbian Learning based Orthogonal Projection for Continual Learning of Spiking Neural Networks](https://mlsanity.com/paper/2402.11984/) (2024)
- [Synaptic Plasticity Models and Bio-Inspired Unsupervised Deep Learning: A Survey](https://mlsanity.com/paper/2307.16236/) (2023)
- [Correlations Are Ruining Your Gradient Descent](https://mlsanity.com/paper/2407.10780/) (2024)
- [Normative framework for deriving neural networks with multi-compartmental neurons and non-Hebbian plasticity](https://mlsanity.com/paper/2302.10051/) (2023)
- [Biologically-inspired Semi-supervised Semantic Segmentation for Biomedical Imaging](https://mlsanity.com/paper/2412.03192/) (2024)
- [A Similarity-preserving Neural Network Trained on Transformed Images Recapitulates Salient Features of the Fly Motion Detection Circuit](https://mlsanity.com/paper/2102.05503/) (2021)
- [PyTorch-Hebbian: facilitating local learning in a deep learning framework](https://mlsanity.com/paper/2102.00428/) (2021)
- [An online algorithm for contrastive Principal Component Analysis](https://mlsanity.com/paper/2211.07723/) (2022)
- [Unsupervised 3D Object Learning through Neuron Activity aware Plasticity](https://mlsanity.com/paper/2302.11622/) (2023)
- [Duality Principle and Biologically Plausible Learning: Connecting the Representer Theorem and Hebbian Learning](https://mlsanity.com/paper/2309.16687/) (2023)
- [Training Large Neural Networks With Low-Dimensional Error Feedback](https://mlsanity.com/paper/2502.20580/) (2025)
- [Curl Descent: Non-Gradient Learning Dynamics with Sign-Diverse Plasticity](https://mlsanity.com/paper/2510.02765/) (2025)
- [A Neural Network with Local Learning Rules for Minor Subspace Analysis](https://mlsanity.com/paper/2102.05501/) (2021)
- [Oja's plasticity rule overcomes several challenges of training neural networks under biological constraints](https://mlsanity.com/paper/2408.08408/) (2024)
- [Credit Assignment via Neural Manifold Noise Correlation](https://mlsanity.com/paper/2601.02636/) (2026)
- [Energy-Based Dynamical Models for Neurocomputation, Learning, and Optimization](https://mlsanity.com/paper/2604.05042/) (2026)
- [Kernel similarity matching with Hebbian neural networks](https://mlsanity.com/paper/2204.07475/) (2022)
- [Neuronal Temporal Filters as Normal Mode Extractors](https://mlsanity.com/paper/2401.03248/) (2024)
- [Similarity Matching Networks: Hebbian Learning and Convergence Over Multiple Time Scales](https://mlsanity.com/paper/2506.06134/) (2025)
- [Rethinking Hebbian Principle: Low-Dimensional Structural Projection for Unsupervised Learning](https://mlsanity.com/paper/2510.14810/) (2025)

## Abstract (arXiv)

Neural network models of early sensory processing typically reduce the dimensionality of streaming input data. Such networks learn the principal subspace, in the sense of principal component analysis (PCA), by adjusting synaptic weights according to activity-dependent learning rules. When derived from a principled cost function these rules are nonlocal and hence biologically implausible. At the same time, biologically plausible local rules have been postulated rather than derived from a principled cost function. Here, to bridge this gap, we derive a biologically plausible network for subspace learning on streaming data by minimizing a principled cost function. In a departure from previous work, where cost was quantified by the representation, or reconstruction, error, we adopt a multidimensional scaling (MDS) cost function for streaming data. The resulting algorithm relies only on biologically plausible Hebbian and anti-Hebbian local learning rules. In a stochastic setting, synaptic weights converge to a stationary state which projects the input data onto the principal subspace. If the data are generated by a nonstationary distribution, the network can track the principal subspace. Thus, our result makes a step towards an algorithmic theory of neural computation.