mlsanityData to 7 Oct 2026

Hebbian Deep Learning Without Feedback

arXiv 2209.11883 ↗First posted 23 Sep 2022cs.NE cs.LG q-bio.NCcited by 61

Not on the map. Why?

Abstract (arXiv)

Recent approximations to backpropagation (BP) have mitigated many of BP's computational inefficiencies and incompatibilities with biology, but important limitations still remain. Moreover, the approximations significantly decrease accuracy in benchmarks, suggesting that an entirely different approach may be more fruitful. Here, grounded on recent theory for Hebbian learning in soft winner-take-all networks, we present multilayer SoftHebb, i.e. an algorithm that trains deep neural networks, without any feedback, target, or error signals. As a result, it achieves efficiency by avoiding weight transport, non-local plasticity, time-locking of layer updates, iterative equilibria, and (self-) supervisory or other feedback signals -- which were necessary in other approaches. Its increased efficiency and biological compatibility do not trade off accuracy compared to state-of-the-art bio-plausible learning, but rather improve it. With up to five hidden layers and an added linear classifier, accuracies on MNIST, CIFAR-10, STL-10, and ImageNet, respectively reach 99.4%, 80.3%, 76.2%, and 27.3%. In conclusion, SoftHebb shows with a radically different approach from BP that Deep Learning over few layers may be plausible in the brain and increases the accuracy of bio-plausible machine learning. Code is available at https://github.com/NeuromorphicComputing/SoftHebb.

Key facts. Cited by 61 papers in the map (data through 7 Oct 2026). Builds on 41 of its 91 references that are in the map. First posted on arXiv 23 Sep 2022.

Builds on

41 of 91 references.

References · 41

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  4. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  5. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  6. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  7. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  8. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  9. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,845
  10. 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,841
  11. 2014Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Imagescited by 774
  12. 2019Big Transfer (BiT): General Visual Representation Learningcited by 766
29 more
  1. 2016Equilibrium Propagation: Bridging the Gap Between Energy-Based Models and Backpropagationcited by 251
  2. 2016Direct Feedback Alignment Provides Learning in Deep Neural Networkscited by 240
  3. 2016Dense Associative Memory for Pattern Recognitioncited by 230
  4. 2020Representation Learning via Invariant Causal Mechanismscited by 197
  5. 2016Decoupled Neural Interfaces using Synthetic Gradientscited by 135
  6. 2019Training Neural Networks with Local Error Signalscited by 135
  7. 2018Assessing the Scalability of Biologically-Motivated Deep Learning Algorithms and Architecturescited by 100
  8. 2020Predictive Coding Approximates Backprop along Arbitrary Computation Graphscited by 89
  9. 2019Putting An End to End-to-End: Gradient-Isolated Learning of Representationscited by 85
  10. 2022Pushing the limits of self-supervised ResNets: Can we outperform supervised learning without labels on ImageNet?cited by 67
  11. 2020Scaling Equilibrium Propagation to Deep ConvNets by Drastically Reducing its Gradient Estimator Biascited by 60
  12. 2020The Unreasonable Effectiveness of Deep Learning in Artificial Intelligencecited by 57
  13. 2021Improving Transferability of Representations via Augmentation-Aware Self-Supervisioncited by 48
  14. 2020Local plasticity rules can learn deep representations using self-supervised contrastive predictionscited by 47
  15. 2018Unsupervised Learning by Competing Hidden Unitscited by 44
  16. 2017Understanding Synthetic Gradients and Decoupled Neural Interfacescited by 35
  17. 2021SoftHebb: Bayesian Inference in Unsupervised Hebbian Soft Winner-Take-All Networkscited by 29
  18. 2021Hebbian Semi-Supervised Learning in a Sample Efficiency Settingcited by 19
  19. 2022Single-phase deep learning in cortico-cortical networkscited by 19
  20. 2019Local Unsupervised Learning for Image Analysiscited by 16
  21. 2021Towards Biologically Plausible Convolutional Networkscited by 13
  22. 2015A Normative Theory of Adaptive Dimensionality Reduction in Neural Networkscited by 12
  23. 2016Bio-Inspired Spiking Convolutional Neural Network using Layer-wise Sparse Coding and STDP Learningcited by 9
  24. 2017Learning in the Machine: the Symmetries of the Deep Learning Channelcited by 9
  25. 2018Bayesian filtering unifies adaptive and non-adaptive neural network optimization methodscited by 9
  26. 2015Convolutional Clustering for Unsupervised Learningcited by 7
  27. 2021Introducing "Neuromorphic Computing and Engineering"cited by 6
  28. 2022Short-Term Plasticity Neurons Learning to Learn and Forgetcited by 5
  29. 2020CSNNs: Unsupervised, Backpropagation-free Convolutional Neural Networks for Representation Learningcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 3; 2023: 11; 2024: 18; 2025: 16; 2026: 13.

61 papers.

Sort:

  1. 2024Benchmarking Predictive Coding Networks -- Made Simplecited by 29
  2. 2023Blockwise Self-Supervised Learning at Scalecited by 22
  3. 2022A Stable, Fast, and Fully Automatic Learning Algorithm for Predictive Coding Networkscited by 21
  4. 2024Self-Contrastive Forward-Forward Algorithmcited by 15
  5. 2023Forward Learning with Top-Down Feedback: Empirical and Analytical Characterizationcited by 13
  6. 2023Brain-Inspired Machine Intelligence: A Survey of Neurobiologically-Plausible Credit Assignmentcited by 13
  7. 2023Training Convolutional Neural Networks with the Forward-Forward algorithmcited by 13
  8. 2022Self-Supervised Learning Through Efference Copiescited by 9
  9. 2023Spiking Neural Networks and Bio-Inspired Supervised Deep Learning: A Surveycited by 8
  10. 2023Improving equilibrium propagation without weight symmetry through Jacobian homeostasiscited by 8
  11. 2023Memoria: Resolving Fateful Forgetting Problem through Human-Inspired Memory Architecturecited by 8
  12. 2024End-to-End Training Induces Information Bottleneck through Layer-Role Differentiation: A Comparative Analysis with Layer-wise Trainingcited by 8
38 more
  1. 2024Scaling Supervised Local Learning with Augmented Auxiliary Networkscited by 8
  2. 2025NoProp: Training Neural Networks without Full Back-propagation or Full Forward-propagationcited by 8
  3. 2023Synaptic Plasticity Models and Bio-Inspired Unsupervised Deep Learning: A Surveycited by 7
  4. 2022Activation Learning by Local Competitionscited by 6
  5. 2024A Review of Neuroscience-Inspired Machine Learningcited by 6
  6. 2024LLS: Local Learning Rule for Deep Neural Networks Inspired by Neural Activity Synchronizationcited by 5
  7. 2024MLAAN: Scaling Supervised Local Learning with Multilaminar Leap Augmented Auxiliary Networkcited by 5
  8. 2024HPFF: Hierarchical Locally Supervised Learning with Patch Feature Fusioncited by 5
  9. 2023The Integrated Forward-Forward Algorithm: Integrating Forward-Forward and Shallow Backpropagation With Local Lossescited by 4
  10. 2024Towards Interpretable Deep Local Learning with Successive Gradient Reconciliationcited by 4
  11. 2024Neuronal Competition Groups with Supervised STDP for Spike-Based Classificationcited by 4
  12. 2024Biologically-inspired Semi-supervised Semantic Segmentation for Biomedical Imagingcited by 4
  13. 2025Stochastic Forward-Forward Learning through Representational Dimensionality Compressioncited by 4
  14. 2024LightFF: Lightweight Inference for Forward-Forward Algorithmcited by 3
  15. 2024FLOPS: Forward Learning with OPtimal Samplingcited by 3
  16. 2024Temporal-Difference Learning Using Distributed Error Signalscited by 3
  17. 2026Faster Predictive Coding Networks via Better Initializationcited by 3
  18. 2023Efficient and Flexible Neural Network Training through Layer-wise Feedback Propagationcited by 2
  19. 2024Two Tales of Single-Phase Contrastive Hebbian Learningcited by 2
  20. 2025OscNet: Machine Learning on CMOS Oscillator Networkscited by 2
  21. 2025From Neurons to Computation: Biological Reservoir Computing for Pattern Recognitioncited by 2
  22. 2025Edge Intelligence with Spiking Neural Networkscited by 2
  23. 2025Toward Practical Equilibrium Propagation: Brain-inspired Recurrent Neural Network with Feedback Regulation and Residual Connectionscited by 2
  24. 2026Covariance-Aware Goodness for Scalable Forward-Forward Learningcited by 2
  25. 2024Decoupled Vertical Federated Learning for Practical Training on Vertically Partitioned Datacited by 1
  26. 2024Forward Learning of Graph Neural Networkscited by 1
  27. 2024Unsupervised End-to-End Training with a Self-Defined Targetcited by 1
  28. 2025Stochastic Layer-wise Learning: Scalable and Efficient Alternative to Backpropagationcited by 1
  29. 2025NM-Hebb: Coupling Local Hebbian Plasticity with Metric Learning for More Accurate and Interpretable CNNscited by 1
  30. 2025From Neural Activity to Computation: Biological Reservoirs for Pattern Recognition in Digit Classificationcited by 1
  31. 2026Neuro-Inspired Visual Pattern Recognition via Biological Reservoir Computingcited by 1
  32. 2026Local learning for stable backpropagation-free neural network training towards physical learningcited by 1
  33. 2026Incremental learning for audio classification with Hebbian Deep Neural Networkscited by 1
  34. 2026Hyperspherical Forward-Forward with Prototypical Representationscited by 1
  35. 2026HCL-FF: Hierarchical and Contrastive Learning for Forward-Forward Algorithmcited by 1
  36. 2023Towards Unsupervised Representation Learning: Learning, Evaluating and Transferring Visual Representationscited by 0
  37. 2025Classifying States of the Hopfield Network with Improved Accuracy, Generalization, and Interpretabilitycited by 0
  38. 2025Energy-Efficient Supervised Learning with a Binary Stochastic Forward-Forward Algorithmcited by 0

50 most cited of 61 All 61 (JSON)