# Biologically plausible deep learning -- but how far can we go with shallow networks?

arXiv: https://arxiv.org/abs/1905.04101 · HTML: https://mlsanity.com/paper/1905.04101/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Feb 2019.

## Built on by (most cited first, 25 of 25)

- [SoftHebb: Bayesian Inference in Unsupervised Hebbian Soft Winner-Take-All Networks](https://mlsanity.com/paper/2107.05747/) (2021)
- [Reverse Differentiation via Predictive Coding](https://mlsanity.com/paper/2103.04689/) (2021)
- [Feature Learning in L2-regularized DNNs: Attraction/Repulsion and Sparsity](https://mlsanity.com/paper/2205.15809/) (2022)
- [Predictive Coding Can Do Exact Backpropagation on Convolutional and Recurrent Neural Networks](https://mlsanity.com/paper/2103.03725/) (2021)
- [BSNN: Towards Faster and Better Conversion of Artificial Neural Networks to Spiking Neural Networks with Bistable Neurons](https://mlsanity.com/paper/2105.12917/) (2021)
- [Spiking-GAN: A Spiking Generative Adversarial Network Using Time-To-First-Spike Coding](https://mlsanity.com/paper/2106.15420/) (2021)
- [Linear Leaky-Integrate-and-Fire Neuron Model Based Spiking Neural Networks and Its Mapping Relationship to Deep Neural Networks](https://mlsanity.com/paper/2207.04889/) (2022)
- [Spiking Neural Networks and Bio-Inspired Supervised Deep Learning: A Survey](https://mlsanity.com/paper/2307.16235/) (2023)
- [Synaptic Plasticity Models and Bio-Inspired Unsupervised Deep Learning: A Survey](https://mlsanity.com/paper/2307.16236/) (2023)
- [BioLeaF: A Bio-plausible Learning Framework for Training of Spiking Neural Networks](https://mlsanity.com/paper/2111.13188/) (2021)
- [Training Deep Spiking Auto-encoders without Bursting or Dying Neurons through Regularization](https://mlsanity.com/paper/2109.11045/) (2021)
- [From Neurons to Computation: Biological Reservoir Computing for Pattern Recognition](https://mlsanity.com/paper/2505.03510/) (2025)
- [Using brain inspired principles to unsupervisedly learn good representations for visual pattern recognition](https://mlsanity.com/paper/2104.14970/) (2021)
- [BioLCNet: Reward-modulated Locally Connected Spiking Neural Networks](https://mlsanity.com/paper/2109.05539/) (2021)
- [A Computational Framework of Cortical Microcircuits Approximates Sign-concordant Random Backpropagation](https://mlsanity.com/paper/2205.07292/) (2022)
- [Artificial Human Intelligence: The role of Humans in the Development of Next Generation AI](https://mlsanity.com/paper/2409.16001/) (2024)
- [From Neural Activity to Computation: Biological Reservoirs for Pattern Recognition in Digit Classification](https://mlsanity.com/paper/2510.05637/) (2025)
- [Neuro-Inspired Visual Pattern Recognition via Biological Reservoir Computing](https://mlsanity.com/paper/2602.05737/) (2026)
- [Self-supervised local learning rules learn the hidden hierarchical structure of high-dimensional data](https://mlsanity.com/paper/2605.18557/) (2026)
- [A Computational Model of Representation Learning in the Brain Cortex, Integrating Unsupervised and Reinforcement Learning](https://mlsanity.com/paper/2106.03688/) (2021)
- [Multi-level Data Representation For Training Deep Helmholtz Machines](https://mlsanity.com/paper/2210.14855/) (2022)
- [Synaptic Dynamics Realize First-order Adaptive Learning and Weight Symmetry](https://mlsanity.com/paper/2212.09440/) (2022)
- [Noise-based reward-modulated learning](https://mlsanity.com/paper/2503.23972/) (2025)
- [Learning in Spiking Neural Networks with a Calcium-based Hebbian Rule for Spike-timing-dependent Plasticity](https://mlsanity.com/paper/2504.06796/) (2025)
- [Hebbian Learning with Global Direction](https://mlsanity.com/paper/2601.21367/) (2026)

## Abstract (arXiv)

Training deep neural networks with the error backpropagation algorithm is considered implausible from a biological perspective. Numerous recent publications suggest elaborate models for biologically plausible variants of deep learning, typically defining success as reaching around 98% test accuracy on the MNIST data set. Here, we investigate how far we can go on digit (MNIST) and object (CIFAR10) classification with biologically plausible, local learning rules in a network with one hidden layer and a single readout layer. The hidden layer weights are either fixed (random or random Gabor filters) or trained with unsupervised methods (PCA, ICA or Sparse Coding) that can be implemented by local learning rules. The readout layer is trained with a supervised, local learning rule. We first implement these models with rate neurons. This comparison reveals, first, that unsupervised learning does not lead to better performance than fixed random projections or Gabor filters for large hidden layers. Second, networks with localized receptive fields perform significantly better than networks with all-to-all connectivity and can reach backpropagation performance on MNIST. We then implement two of the networks - fixed, localized, random & random Gabor filters in the hidden layer - with spiking leaky integrate-and-fire neurons and spike timing dependent plasticity to train the readout layer. These spiking models achieve > 98.2% test accuracy on MNIST, which is close to the performance of rate networks with one hidden layer trained with backpropagation. The performance of our shallow network models is comparable to most current biologically plausible models of deep learning. Furthermore, our results with a shallow spiking network provide an important reference and suggest the use of datasets other than MNIST for testing the performance of future models of biologically plausible deep learning.