# FastHebb: Scaling Hebbian Training of Deep Neural Networks to ImageNet Level

arXiv: https://arxiv.org/abs/2207.03172 · HTML: https://mlsanity.com/paper/2207.03172/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 38 references that are in the map.
- First posted on arXiv 7 Jul 2022.

## Builds on (most cited first, 8 of 8)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Semi-Supervised Learning with Deep Generative Models](https://mlsanity.com/paper/1406.5298/) (2014)
- [Label Propagation for Deep Semi-supervised Learning](https://mlsanity.com/paper/1904.04717/) (2019)
- [Unsupervised Learning by Competing Hidden Units](https://mlsanity.com/paper/1806.10181/) (2018)
- [Hebbian Semi-Supervised Learning in a Sample Efficiency Setting](https://mlsanity.com/paper/2103.09002/) (2021)
- [Augmenting Supervised Neural Networks with Unsupervised Objectives for Large-scale Image Classification](https://mlsanity.com/paper/1606.06582/) (2016)
- [Assessing Pattern Recognition Performance of Neuronal Cultures through Accurate Simulation](https://mlsanity.com/paper/2012.10355/) (2020)
- [Deep Features for CBIR with Scarce Data using Hebbian Learning](https://mlsanity.com/paper/2205.08935/) (2022)

## Built on by (most cited first, 11 of 11)

- [Self-Contrastive Forward-Forward Algorithm](https://mlsanity.com/paper/2409.11593/) (2024)
- [Spiking Neural Networks and Bio-Inspired Supervised Deep Learning: A Survey](https://mlsanity.com/paper/2307.16235/) (2023)
- [Synaptic Plasticity Models and Bio-Inspired Unsupervised Deep Learning: A Survey](https://mlsanity.com/paper/2307.16236/) (2023)
- [Biologically-inspired Semi-supervised Semantic Segmentation for Biomedical Imaging](https://mlsanity.com/paper/2412.03192/) (2024)
- [Advancing the Biological Plausibility and Efficacy of Hebbian Convolutional Neural Networks](https://mlsanity.com/paper/2501.17266/) (2025)
- [From Neurons to Computation: Biological Reservoir Computing for Pattern Recognition](https://mlsanity.com/paper/2505.03510/) (2025)
- [NM-Hebb: Coupling Local Hebbian Plasticity with Metric Learning for More Accurate and Interpretable CNNs](https://mlsanity.com/paper/2508.19896/) (2025)
- [From Neural Activity to Computation: Biological Reservoirs for Pattern Recognition in Digit Classification](https://mlsanity.com/paper/2510.05637/) (2025)
- [Memory Networks: Towards Fully Biologically Plausible Learning](https://mlsanity.com/paper/2409.17282/) (2024)
- [Neural Architecture Search with Mixed Bio-inspired Learning Rules](https://mlsanity.com/paper/2507.13485/) (2025)
- [Hebbian Learning with Global Direction](https://mlsanity.com/paper/2601.21367/) (2026)

## Abstract (arXiv)

Learning algorithms for Deep Neural Networks are typically based on supervised end-to-end Stochastic Gradient Descent (SGD) training with error backpropagation (backprop). Backprop algorithms require a large number of labelled training samples to achieve high performance. However, in many realistic applications, even if there is plenty of image samples, very few of them are labelled, and semi-supervised sample-efficient training strategies have to be used. Hebbian learning represents a possible approach towards sample efficient training; however, in current solutions, it does not scale well to large datasets. In this paper, we present FastHebb, an efficient and scalable solution for Hebbian learning which achieves higher efficiency by 1) merging together update computation and aggregation over a batch of inputs, and 2) leveraging efficient matrix multiplication algorithms on GPU. We validate our approach on different computer vision benchmarks, in a semi-supervised learning scenario. FastHebb outperforms previous solutions by up to 50 times in terms of training speed, and notably, for the first time, we are able to bring Hebbian algorithms to ImageNet scale.