# Enabling Spike-based Backpropagation for Training Deep Neural Network Architectures

arXiv: https://arxiv.org/abs/1903.06379 · HTML: https://mlsanity.com/paper/1903.06379/

## Key facts

- Cited by 123 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Mar 2019.

## Built on by (most cited first, 30 of 123)

- [Spikformer: When Spiking Neural Network Meets Transformer](https://mlsanity.com/paper/2209.15425/) (2022)
- [SpikingJelly: An open-source machine learning infrastructure platform for spike-based intelligence](https://mlsanity.com/paper/2310.16620/) (2023)
- [Recurrent Vision Transformers for Object Detection with Event Cameras](https://mlsanity.com/paper/2212.05598/) (2022)
- [Training High-Performance Low-Latency Spiking Neural Networks by Differentiation on Spike Representation](https://mlsanity.com/paper/2205.00459/) (2022)
- [HIRE-SNN: Harnessing the Inherent Robustness of Energy-Efficient Deep Spiking Neural Networks by Training with Crafted Input Noise](https://mlsanity.com/paper/2110.11417/) (2021)
- [Event-based Video Reconstruction via Potential-assisted Spiking Neural Network](https://mlsanity.com/paper/2201.10943/) (2022)
- [Neural Architecture Search for Spiking Neural Networks](https://mlsanity.com/paper/2201.10355/) (2022)
- [Fast-SNN: Fast Spiking Neural Network by Converting Quantized ANN](https://mlsanity.com/paper/2305.19868/) (2023)
- [Pruning of Deep Spiking Neural Networks through Gradient Rewiring](https://mlsanity.com/paper/2105.04916/) (2021)
- [Optimized Potential Initialization for Low-latency Spiking Neural Networks](https://mlsanity.com/paper/2202.01440/) (2022)
- [Training Feedback Spiking Neural Networks by Implicit Differentiation on the Equilibrium State](https://mlsanity.com/paper/2109.14247/) (2021)
- [AutoSNN: Towards Energy-Efficient Spiking Neural Networks](https://mlsanity.com/paper/2201.12738/) (2022)
- [Membrane Potential Batch Normalization for Spiking Neural Networks](https://mlsanity.com/paper/2308.08359/) (2023)
- [A Fully Spiking Hybrid Neural Network for Energy-Efficient Object Detection](https://mlsanity.com/paper/2104.10719/) (2021)
- [Spikformer V2: Join the High Accuracy Club on ImageNet with an SNN Ticket](https://mlsanity.com/paper/2401.02020/) (2024)
- [EventDrop: data augmentation for event-based learning](https://mlsanity.com/paper/2106.05836/) (2021)
- [RMP-Loss: Regularizing Membrane Potential Distribution for Spiking Neural Networks](https://mlsanity.com/paper/2308.06787/) (2023)
- [Visual Explanations from Spiking Neural Networks using Interspike Intervals](https://mlsanity.com/paper/2103.14441/) (2021)
- [Exploring Lottery Ticket Hypothesis in Spiking Neural Networks](https://mlsanity.com/paper/2207.01382/) (2022)
- [Training Full Spike Neural Networks via Auxiliary Accumulation Pathway](https://mlsanity.com/paper/2301.11929/) (2023)
- [Rate Coding or Direct Coding: Which One is Better for Accurate, Robust, and Energy-efficient Spiking Neural Networks?](https://mlsanity.com/paper/2202.03133/) (2022)
- [Exploring Temporal Information Dynamics in Spiking Neural Networks](https://mlsanity.com/paper/2211.14406/) (2022)
- [Backpropagation with Biologically Plausible Spatio-Temporal Adjustment For Training Deep Spiking Neural Networks](https://mlsanity.com/paper/2110.08858/) (2021)
- [Real Spike: Learning Real-valued Spikes for Spiking Neural Networks](https://mlsanity.com/paper/2210.06686/) (2022)
- [Multi-Level Firing with Spiking DS-ResNet: Enabling Better and Deeper Directly-Trained Spiking Neural Networks](https://mlsanity.com/paper/2210.06386/) (2022)
- [Spiking Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2306.17046/) (2023)
- [One Timestep is All You Need: Training Spiking Neural Networks with Ultra Low Latency](https://mlsanity.com/paper/2110.05929/) (2021)
- [PrivateSNN: Privacy-Preserving Spiking Neural Networks](https://mlsanity.com/paper/2104.03414/) (2021)
- [Spiking Neural Networks for Visual Place Recognition via Weighted Neuronal Assignments](https://mlsanity.com/paper/2109.06452/) (2021)
- [BSNN: Towards Faster and Better Conversion of Artificial Neural Networks to Spiking Neural Networks with Bistable Neurons](https://mlsanity.com/paper/2105.12917/) (2021)

## Abstract (arXiv)

Spiking Neural Networks (SNNs) have recently emerged as a prominent neural computing paradigm. However, the typical shallow SNN architectures have limited capacity for expressing complex representations while training deep SNNs using input spikes has not been successful so far. Diverse methods have been proposed to get around this issue such as converting off-the-shelf trained deep Artificial Neural Networks (ANNs) to SNNs. However, the ANN-SNN conversion scheme fails to capture the temporal dynamics of a spiking system. On the other hand, it is still a difficult problem to directly train deep SNNs using input spike events due to the discontinuous, non-differentiable nature of the spike generation function. To overcome this problem, we propose an approximate derivative method that accounts for the leaky behavior of LIF neurons. This method enables training deep convolutional SNNs directly (with input spike events) using spike-based backpropagation. Our experiments show the effectiveness of the proposed spike-based learning on deep networks (VGG and Residual architectures) by achieving the best classification accuracies in MNIST, SVHN and CIFAR-10 datasets compared to other SNNs trained with a spike-based learning. Moreover, we analyze sparse event-based computations to demonstrate the efficacy of the proposed SNN training method for inference operation in the spiking domain.