# Neuromorphic Deep Learning Machines

arXiv: https://arxiv.org/abs/1612.05596 · HTML: https://mlsanity.com/paper/1612.05596/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Dec 2016.

## Built on by (most cited first, 30 of 38)

- [Training Spiking Neural Networks Using Lessons From Deep Learning](https://mlsanity.com/paper/2109.12894/) (2021)
- [RMP-Loss: Regularizing Membrane Potential Distribution for Spiking Neural Networks](https://mlsanity.com/paper/2308.06787/) (2023)
- [Signal Propagation: A Framework for Learning and Inference In a Forward Pass](https://mlsanity.com/paper/2204.01723/) (2022)
- [Bottom-up and top-down approaches for the design of neuromorphic processing systems: Tradeoffs and synergies between natural and artificial intelligence](https://mlsanity.com/paper/2106.01288/) (2021)
- [Representation Learning for Event-based Visuomotor Policies](https://mlsanity.com/paper/2103.00806/) (2021)
- [The Backpropagation Algorithm Implemented on Spiking Neuromorphic Hardware](https://mlsanity.com/paper/2106.07030/) (2021)
- [Self-Contrastive Forward-Forward Algorithm](https://mlsanity.com/paper/2409.11593/) (2024)
- [Learning to Time-Decode in Spiking Neural Networks Through the Information Bottleneck](https://mlsanity.com/paper/2106.01177/) (2021)
- [Unleashing the Potential of Spiking Neural Networks by Dynamic Confidence](https://mlsanity.com/paper/2303.10276/) (2023)
- [Meta-learning Spiking Neural Networks with Surrogate Gradient Descent](https://mlsanity.com/paper/2201.10777/) (2022)
- [SPIDE: A Purely Spike-based Method for Training Feedback Spiking Neural Networks](https://mlsanity.com/paper/2302.00232/) (2023)
- [Supervised training of spiking neural networks for robust deployment on mixed-signal neuromorphic processors](https://mlsanity.com/paper/2102.06408/) (2021)
- [Low-Variance Forward Gradients using Direct Feedback Alignment and Momentum](https://mlsanity.com/paper/2212.07282/) (2022)
- [Spike time displacement based error backpropagation in convolutional spiking neural networks](https://mlsanity.com/paper/2108.13621/) (2021)
- [A Review of Neuroscience-Inspired Machine Learning](https://mlsanity.com/paper/2403.18929/) (2024)
- [Paired Competing Neurons Improving STDP Supervised Local Learning In Spiking Neural Networks](https://mlsanity.com/paper/2308.02194/) (2023)
- [Sparse-firing regularization methods for spiking neural networks with time-to-first spike coding](https://mlsanity.com/paper/2307.13007/) (2023)
- [Unlocking Deep Learning: A BP-Free Approach for Parallel Block-Wise Training of Neural Networks](https://mlsanity.com/paper/2312.13311/) (2023)
- [Online Pseudo-Zeroth-Order Training of Neuromorphic Spiking Neural Networks](https://mlsanity.com/paper/2407.12516/) (2024)
- [Foveal-pit inspired filtering of DVS spike response](https://mlsanity.com/paper/2105.14331/) (2021)
- [BioGrad: Biologically Plausible Gradient-Based Learning for Spiking Neural Networks](https://mlsanity.com/paper/2110.14092/) (2021)
- [SQUAT: Stateful Quantization-Aware Training in Recurrent Spiking Neural Networks](https://mlsanity.com/paper/2404.19668/) (2024)
- [Training Spiking Neural Networks via Augmented Direct Feedback Alignment](https://mlsanity.com/paper/2409.07776/) (2024)
- [Implementing a foveal-pit inspired filter in a Spiking Convolutional Neural Network: a preliminary study](https://mlsanity.com/paper/2105.14326/) (2021)
- [Including STDP to eligibility propagation in multi-layer recurrent spiking neural networks](https://mlsanity.com/paper/2201.07602/) (2022)
- [A Computational Framework of Cortical Microcircuits Approximates Sign-concordant Random Backpropagation](https://mlsanity.com/paper/2205.07292/) (2022)
- [TACOS: Task Agnostic Continual Learning in Spiking Neural Networks](https://mlsanity.com/paper/2409.00021/) (2024)
- [Harnessing Nonidealities in Analog In-Memory Computing Circuits: A Physical Modeling Approach for Neuromorphic Systems](https://mlsanity.com/paper/2412.09010/) (2024)
- [Feedback Alignment Meets Low-Rank Manifolds: A Structured Recipe for Local Learning](https://mlsanity.com/paper/2510.25594/) (2025)
- [Visual Sensation and Perception Computational Models for Deep Learning: State of the art, Challenges and Prospects](https://mlsanity.com/paper/2109.03391/) (2021)

## Abstract (arXiv)

An ongoing challenge in neuromorphic computing is to devise general and computationally efficient models of inference and learning which are compatible with the spatial and temporal constraints of the brain. One increasingly popular and successful approach is to take inspiration from inference and learning algorithms used in deep neural networks. However, the workhorse of deep learning, the gradient descent Back Propagation (BP) rule, often relies on the immediate availability of network-wide information stored with high-precision memory, and precise operations that are difficult to realize in neuromorphic hardware. Remarkably, recent work showed that exact backpropagated weights are not essential for learning deep representations. Random BP replaces feedback weights with random ones and encourages the network to adjust its feed-forward weights to learn pseudo-inverses of the (random) feedback weights. Building on these results, we demonstrate an event-driven random BP (eRBP) rule that uses an error-modulated synaptic plasticity for learning deep representations in neuromorphic computing hardware. The rule requires only one addition and two comparisons for each synaptic weight using a two-compartment leaky Integrate & Fire (I&F) neuron, making it very suitable for implementation in digital or mixed-signal neuromorphic hardware. Our results show that using eRBP, deep representations are rapidly learned, achieving nearly identical classification accuracies compared to artificial neural network simulations on GPUs, while being robust to neural and synaptic state quantizations during learning.