# Towards Biologically Plausible Deep Learning

arXiv: https://arxiv.org/abs/1502.04156 · HTML: https://mlsanity.com/paper/1502.04156/

## Key facts

- Cited by 66 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Feb 2015.

## Built on by (most cited first, 30 of 66)

- [Training Spiking Neural Networks Using Lessons From Deep Learning](https://mlsanity.com/paper/2109.12894/) (2021)
- [Gradients without Backpropagation](https://mlsanity.com/paper/2202.08587/) (2022)
- [Revisiting Locally Supervised Learning: an Alternative to End-to-end Training](https://mlsanity.com/paper/2101.10832/) (2021)
- [Masked Image Modeling with Local Multi-Scale Reconstruction](https://mlsanity.com/paper/2303.05251/) (2023)
- [SoftHebb: Bayesian Inference in Unsupervised Hebbian Soft Winner-Take-All Networks](https://mlsanity.com/paper/2107.05747/) (2021)
- [Brain-inspired Computational Intelligence via Predictive Coding](https://mlsanity.com/paper/2308.07870/) (2023)
- [Credit Assignment in Neural Networks through Deep Feedback Control](https://mlsanity.com/paper/2106.07887/) (2021)
- [Bottom-up and top-down approaches for the design of neuromorphic processing systems: Tradeoffs and synergies between natural and artificial intelligence](https://mlsanity.com/paper/2106.01288/) (2021)
- [Feature Learning in L2-regularized DNNs: Attraction/Repulsion and Sparsity](https://mlsanity.com/paper/2205.15809/) (2022)
- [Brain-Inspired Machine Intelligence: A Survey of Neurobiologically-Plausible Credit Assignment](https://mlsanity.com/paper/2312.09257/) (2023)
- [How to Train Your Wide Neural Network Without Backprop: An Input-Weight Alignment Perspective](https://mlsanity.com/paper/2106.08453/) (2021)
- [Backprop-Free Reinforcement Learning with Active Neural Generative Coding](https://mlsanity.com/paper/2107.07046/) (2021)
- [Meta-Learning Bidirectional Update Rules](https://mlsanity.com/paper/2104.04657/) (2021)
- [Momentum Auxiliary Network for Supervised Local Learning](https://mlsanity.com/paper/2407.05623/) (2024)
- [Online Spatio-Temporal Learning with Target Projection](https://mlsanity.com/paper/2304.05124/) (2023)
- [Scaling Supervised Local Learning with Augmented Auxiliary Networks](https://mlsanity.com/paper/2402.17318/) (2024)
- [Fast weight programming and linear transformers: from machine learning to neurobiology](https://mlsanity.com/paper/2508.08435/) (2025)
- [Synaptic Plasticity Models and Bio-Inspired Unsupervised Deep Learning: A Survey](https://mlsanity.com/paper/2307.16236/) (2023)
- [Beyond Backpropagation: Optimization with Multi-Tangent Forward Gradients](https://mlsanity.com/paper/2410.17764/) (2024)
- [Bilevel Programs Meet Deep Learning: A Unifying View on Inference Learning Methods](https://mlsanity.com/paper/2105.07231/) (2021)
- [Gradient Forward-Propagation for Large-Scale Temporal Video Modelling](https://mlsanity.com/paper/2106.08318/) (2021)
- [MLAAN: Scaling Supervised Local Learning with Multilaminar Leap Augmented Auxiliary Network](https://mlsanity.com/paper/2406.16633/) (2024)
- [Second-Order Forward-Mode Automatic Differentiation for Optimization](https://mlsanity.com/paper/2408.10419/) (2024)
- [Counter-Current Learning: A Biologically Plausible Dual Network Approach for Deep Learning](https://mlsanity.com/paper/2409.19841/) (2024)
- [Backpropagation-free Spiking Neural Networks with the Forward-Forward Algorithm](https://mlsanity.com/paper/2502.20411/) (2025)
- [SpikePropamine: Differentiable Plasticity in Spiking Neural Networks](https://mlsanity.com/paper/2106.02681/) (2021)
- [Towards Interpretable Deep Local Learning with Successive Gradient Reconciliation](https://mlsanity.com/paper/2406.05222/) (2024)
- [Local Learning Rules for Out-of-Equilibrium Physical Generative Models](https://mlsanity.com/paper/2506.19136/) (2025)
- [Feed-Forward Optimization With Delayed Feedback for Neural Network Training](https://mlsanity.com/paper/2304.13372/) (2023)
- [MAN++: Scaling Momentum Auxiliary Network for Supervised Local Learning in Vision Tasks](https://mlsanity.com/paper/2507.16279/) (2025)

## Abstract (arXiv)

Neuroscientists have long criticised deep learning algorithms as incompatible with current knowledge of neurobiology. We explore more biologically plausible versions of deep representation learning, focusing here mostly on unsupervised learning but developing a learning mechanism that could account for supervised, unsupervised and reinforcement learning. The starting point is that the basic learning rule believed to govern synaptic weight updates (Spike-Timing-Dependent Plasticity) arises out of a simple update rule that makes a lot of sense from a machine learning point of view and can be interpreted as gradient descent on some objective function so long as the neuronal dynamics push firing rates towards better values of the objective function (be it supervised, unsupervised, or reward-driven). The second main idea is that this corresponds to a form of the variational EM algorithm, i.e., with approximate rather than exact posteriors, implemented by neural dynamics. Another contribution of this paper is that the gradients required for updating the hidden states in the above variational interpretation can be estimated using an approximation that only requires propagating activations forward and backward, with pairs of layers learning to form a denoising auto-encoder. Finally, we extend the theory about the probabilistic interpretation of auto-encoders to justify improved sampling schemes based on the generative interpretation of denoising auto-encoders, and we validate all these ideas on generative learning tasks.