# Learning Sparse, Distributed Representations using the Hebbian Principle

arXiv: https://arxiv.org/abs/1611.04228 · HTML: https://mlsanity.com/paper/1611.04228/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Nov 2016.

## Built on by (most cited first, 4 of 4)

- [Hebbian Semi-Supervised Learning in a Sample Efficiency Setting](https://mlsanity.com/paper/2103.09002/) (2021)
- [Spiking Neural Networks and Bio-Inspired Supervised Deep Learning: A Survey](https://mlsanity.com/paper/2307.16235/) (2023)
- [Synaptic Plasticity Models and Bio-Inspired Unsupervised Deep Learning: A Survey](https://mlsanity.com/paper/2307.16236/) (2023)
- [Guiding Sparse Neural Networks with Neurobiological Principles to Elicit Biologically Plausible Representations](https://mlsanity.com/paper/2603.03234/) (2026)

## Abstract (arXiv)

The "fire together, wire together" Hebbian model is a central principle for learning in neuroscience, but surprisingly, it has found limited applicability in modern machine learning. In this paper, we take a first step towards bridging this gap, by developing flavors of competitive Hebbian learning which produce sparse, distributed neural codes using online adaptation with minimal tuning. We propose an unsupervised algorithm, termed Adaptive Hebbian Learning (AHL). We illustrate the distributed nature of the learned representations via output entropy computations for synthetic data, and demonstrate superior performance, compared to standard alternatives such as autoencoders, in training a deep convolutional net on standard image datasets.