# SpikeGrad: An ANN-equivalent Computation Model for Implementing Backpropagation with Spikes

arXiv: https://arxiv.org/abs/1906.00851 · HTML: https://mlsanity.com/paper/1906.00851/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Jun 2019.

## Built on by (most cited first, 16 of 16)

- [Optimal Conversion of Conventional Artificial Neural Networks to Spiking Neural Networks](https://mlsanity.com/paper/2103.00476/) (2021)
- [Online Training Through Time for Spiking Neural Networks](https://mlsanity.com/paper/2210.04195/) (2022)
- [Towards Memory- and Time-Efficient Backpropagation for Training Spiking Neural Networks](https://mlsanity.com/paper/2302.14311/) (2023)
- [Spiking Graph Convolutional Networks](https://mlsanity.com/paper/2205.02767/) (2022)
- [Energy-efficient Knowledge Distillation for Spiking Neural Networks](https://mlsanity.com/paper/2106.07172/) (2021)
- [SPIDE: A Purely Spike-based Method for Training Feedback Spiking Neural Networks](https://mlsanity.com/paper/2302.00232/) (2023)
- [Advancing Training Efficiency of Deep Spiking Neural Networks through Rate-based Backpropagation](https://mlsanity.com/paper/2410.11488/) (2024)
- [Exploiting High Performance Spiking Neural Networks with Efficient Spiking Patterns](https://mlsanity.com/paper/2301.12356/) (2023)
- [Efficient Logit-based Knowledge Distillation of Deep Spiking Neural Networks for Full-Range Timestep Deployment](https://mlsanity.com/paper/2501.15925/) (2025)
- [Spike Accumulation Forwarding for Effective Training of Spiking Neural Networks](https://mlsanity.com/paper/2310.02772/) (2023)
- [Dynamic Spiking Framework for Graph Neural Networks](https://mlsanity.com/paper/2401.05373/) (2023)
- [ChronoSpike: An Adaptive Spiking Graph Neural Network for Dynamic Graphs](https://mlsanity.com/paper/2602.01124/) (2026)
- [A temporally and spatially local spike-based backpropagation algorithm to enable training in hardware](https://mlsanity.com/paper/2207.09755/) (2022)
- [Desire Backpropagation: A Lightweight Training Algorithm for Multi-Layer Spiking Neural Networks based on Spike-Timing-Dependent Plasticity](https://mlsanity.com/paper/2211.05412/) (2022)
- [Membrane Potential Distribution Adjustment and Parametric Surrogate Gradient in Spiking Neural Networks](https://mlsanity.com/paper/2304.13289/) (2023)
- [A Self-Ensemble Inspired Approach for Effective Training of Binary-Weight Spiking Neural Networks](https://mlsanity.com/paper/2508.12609/) (2025)

## Abstract (arXiv)

Event-based neuromorphic systems promise to reduce the energy consumption of deep learning tasks by replacing expensive floating point operations on dense matrices by low power sparse and asynchronous operations on spike events. While these systems can be trained increasingly well using approximations of the back-propagation algorithm, these implementations usually require high precision errors for training and are therefore incompatible with the typical communication infrastructure of neuromorphic circuits. In this work, we analyze how the gradient can be discretized into spike events when training a spiking neural network. To accelerate our simulation, we show that using a special implementation of the integrate-and-fire neuron allows us to describe the accumulated activations and errors of the spiking neural network in terms of an equivalent artificial neural network, allowing us to largely speed up training compared to an explicit simulation of all spike events. This way we are able to demonstrate that even for deep networks, the gradients can be discretized sufficiently well with spikes if the gradient is properly rescaled. This form of spike-based backpropagation enables us to achieve equivalent or better accuracies on the MNIST and CIFAR10 dataset than comparable state-of-the-art spiking neural networks trained with full precision gradients. The algorithm, which we call SpikeGrad, is based on accumulation and comparison operations and can naturally exploit sparsity in the gradient computation, which makes it an interesting choice for a spiking neuromorphic systems with on-chip learning capacities.