# Towards Scalable, Efficient and Accurate Deep Spiking Neural Networks with Backward Residual Connections, Stochastic Softmax and Hybridization

arXiv: https://arxiv.org/abs/1910.13931 · HTML: https://mlsanity.com/paper/1910.13931/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Oct 2019.

## Built on by (most cited first, 5 of 5)

- [A Fully Spiking Hybrid Neural Network for Energy-Efficient Object Detection](https://mlsanity.com/paper/2104.10719/) (2021)
- [Backpropagation with Biologically Plausible Spatio-Temporal Adjustment For Training Deep Spiking Neural Networks](https://mlsanity.com/paper/2110.08858/) (2021)
- [Event-based Synthetic Aperture Imaging with a Hybrid Network](https://mlsanity.com/paper/2103.02376/) (2021)
- [Learning to See Through with Events](https://mlsanity.com/paper/2212.02219/) (2022)
- [SNE: an Energy-Proportional Digital Accelerator for Sparse Event-Based Convolutions](https://mlsanity.com/paper/2204.10687/) (2022)

## Abstract (arXiv)

Spiking Neural Networks (SNNs) may offer an energy-efficient alternative for implementing deep learning applications. In recent years, there have been several proposals focused on supervised (conversion, spike-based gradient descent) and unsupervised (spike timing dependent plasticity) training methods to improve the accuracy of SNNs on large-scale tasks. However, each of these methods suffer from scalability, latency and accuracy limitations. In this paper, we propose novel algorithmic techniques of modifying the SNN configuration with backward residual connections, stochastic softmax and hybrid artificial-and-spiking neuronal activations to improve the learning ability of the training methodologies to yield competitive accuracy, while, yielding large efficiency gains over their artificial counterparts. Note, artificial counterparts refer to conventional deep learning/artificial neural networks. Our techniques apply to VGG/Residual architectures, and are compatible with all forms of training methodologies. Our analysis reveals that the proposed solutions yield near state-of-the-art accuracy with significant energy-efficiency and reduced parameter overhead translating to hardware improvements on complex visual recognition tasks, such as, CIFAR10, Imagenet datatsets.