# Unsupervised Learning by Competing Hidden Units

arXiv: https://arxiv.org/abs/1806.10181 · HTML: https://mlsanity.com/paper/1806.10181/

## Key facts

- Cited by 44 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Jun 2018.

## Built on by (most cited first, 30 of 44)

- [Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networks](https://mlsanity.com/paper/2207.13243/) (2022)
- [Hebbian Deep Learning Without Feedback](https://mlsanity.com/paper/2209.11883/) (2022)
- [Associative Memories via Predictive Coding](https://mlsanity.com/paper/2109.08063/) (2021)
- [SoftHebb: Bayesian Inference in Unsupervised Hebbian Soft Winner-Take-All Networks](https://mlsanity.com/paper/2107.05747/) (2021)
- [Neuro-Modulated Hebbian Learning for Fully Test-Time Adaptation](https://mlsanity.com/paper/2303.00914/) (2023)
- [Hebbian Semi-Supervised Learning in a Sample Efficiency Setting](https://mlsanity.com/paper/2103.09002/) (2021)
- [Can a Fruit Fly Learn Word Embeddings?](https://mlsanity.com/paper/2101.06887/) (2021)
- [Self-Contrastive Forward-Forward Algorithm](https://mlsanity.com/paper/2409.11593/) (2024)
- [Brain-Inspired Machine Intelligence: A Survey of Neurobiologically-Plausible Credit Assignment](https://mlsanity.com/paper/2312.09257/) (2023)
- [FastHebb: Scaling Hebbian Training of Deep Neural Networks to ImageNet Level](https://mlsanity.com/paper/2207.03172/) (2022)
- [Sparse Distributed Memory is a Continual Learner](https://mlsanity.com/paper/2303.11934/) (2023)
- [Is Bio-Inspired Learning Better than Backprop? Benchmarking Bio Learning vs. Backprop](https://mlsanity.com/paper/2212.04614/) (2022)
- [Spiking Neural Networks and Bio-Inspired Supervised Deep Learning: A Survey](https://mlsanity.com/paper/2307.16235/) (2023)
- [Synaptic Plasticity Models and Bio-Inspired Unsupervised Deep Learning: A Survey](https://mlsanity.com/paper/2307.16236/) (2023)
- [Activation Learning by Local Competitions](https://mlsanity.com/paper/2209.13400/) (2022)
- [Training Deep Architectures Without End-to-End Backpropagation: A Survey on the Provably Optimal Methods](https://mlsanity.com/paper/2101.03419/) (2021)
- [Machine learning in and out of equilibrium](https://mlsanity.com/paper/2306.03521/) (2023)
- [Biologically-inspired Semi-supervised Semantic Segmentation for Biomedical Imaging](https://mlsanity.com/paper/2412.03192/) (2024)
- [Advancing the Biological Plausibility and Efficacy of Hebbian Convolutional Neural Networks](https://mlsanity.com/paper/2501.17266/) (2025)
- [Representing Prior Knowledge Using Randomly, Weighted Feature Networks for Visual Relationship Detection](https://mlsanity.com/paper/2111.10686/) (2021)
- [Unsupervised Hebbian Learning on Point Sets in StarCraft II](https://mlsanity.com/paper/2207.12323/) (2022)
- [PyTorch-Hebbian: facilitating local learning in a deep learning framework](https://mlsanity.com/paper/2102.00428/) (2021)
- [Representation Learning in a Decomposed Encoder Design for Bio-inspired Hebbian Learning](https://mlsanity.com/paper/2401.08603/) (2023)
- [OscNet: Machine Learning on CMOS Oscillator Networks](https://mlsanity.com/paper/2502.07192/) (2025)
- [From Neurons to Computation: Biological Reservoir Computing for Pattern Recognition](https://mlsanity.com/paper/2505.03510/) (2025)
- [A feedback control optimizer for online and hardware-aware training of Spiking Neural Networks](https://mlsanity.com/paper/2602.13261/) (2026)
- [Revisiting Gaussian Neurons for Online Clustering with Unknown Number of Clusters](https://mlsanity.com/paper/2205.00920/) (2022)
- [Hebbian Continual Representation Learning](https://mlsanity.com/paper/2207.04874/) (2022)
- [Forecasting Evolution of Clusters in Game Agents with Hebbian Learning](https://mlsanity.com/paper/2209.06904/) (2022)
- [Unsupervised End-to-End Training with a Self-Defined Target](https://mlsanity.com/paper/2403.12116/) (2024)

## Abstract (arXiv)

It is widely believed that the backpropagation algorithm is essential for learning good feature detectors in early layers of artificial neural networks, so that these detectors are useful for the task performed by the higher layers of that neural network. At the same time, the traditional form of backpropagation is biologically implausible. In the present paper we propose an unusual learning rule, which has a degree of biological plausibility, and which is motivated by Hebb's idea that change of the synapse strength should be local - i.e. should depend only on the activities of the pre and post synaptic neurons. We design a learning algorithm that utilizes global inhibition in the hidden layer, and is capable of learning early feature detectors in a completely unsupervised way. These learned lower layer feature detectors can be used to train higher layer weights in a usual supervised way so that the performance of the full network is comparable to the performance of standard feedforward networks trained end-to-end with a backpropagation algorithm.