# Keys to Accurate Feature Extraction Using Residual Spiking Neural Networks

arXiv: https://arxiv.org/abs/2111.05955 · HTML: https://mlsanity.com/paper/2111.05955/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 40 references that are in the map.
- First posted on arXiv 10 Nov 2021.

## Builds on (most cited first, 25 of 25)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Surrogate Gradient Learning in Spiking Neural Networks](https://mlsanity.com/paper/1901.09948/) (2019)
- [BOHB: Robust and Efficient Hyperparameter Optimization at Scale](https://mlsanity.com/paper/1807.01774/) (2018)
- [Spatio-Temporal Backpropagation for Training High-performance Spiking Neural Networks](https://mlsanity.com/paper/1706.02609/) (2017)
- [Incorporating Learnable Membrane Time Constant to Enhance Learning of Spiking Neural Networks](https://mlsanity.com/paper/2007.05785/) (2020)
- [Deep Residual Learning in Spiking Neural Networks](https://mlsanity.com/paper/2102.04159/) (2021)
- [Going Deeper With Directly-Trained Larger Spiking Neural Networks](https://mlsanity.com/paper/2011.05280/) (2020)
- [SLAYER: Spike Layer Error Reassignment in Time](https://mlsanity.com/paper/1810.08646/) (2018)
- [Going Deeper in Spiking Neural Networks: VGG and Residual Architectures](https://mlsanity.com/paper/1802.02627/) (2018)
- [Carbontracker: Tracking and Predicting the Carbon Footprint of Training Deep Learning Models](https://mlsanity.com/paper/2007.03051/) (2020)
- [RMP-SNN: Residual Membrane Potential Neuron for Enabling Deeper High-Accuracy and Low-Latency Spiking Neural Network](https://mlsanity.com/paper/2003.01811/) (2020)
- [Optimal Conversion of Conventional Artificial Neural Networks to Spiking Neural Networks](https://mlsanity.com/paper/2103.00476/) (2021)
- [Enabling Spike-based Backpropagation for Training Deep Neural Network Architectures](https://mlsanity.com/paper/1903.06379/) (2019)
- [Efficient Neuromorphic Signal Processing with Loihi 2](https://mlsanity.com/paper/2111.03746/) (2021)
- [Revisiting Batch Normalization for Training Low-latency Deep Spiking Neural Networks from Scratch](https://mlsanity.com/paper/2010.01729/) (2020)
- [Supervised learning based on temporal coding in spiking neural networks](https://mlsanity.com/paper/1606.08165/) (2016)
- [Convolutional Networks for Fast, Energy-Efficient Neuromorphic Computing](https://mlsanity.com/paper/1603.08270/) (2016)
- [Gradient Descent for Spiking Neural Networks](https://mlsanity.com/paper/1706.04698/) (2017)
- [Progressive Tandem Learning for Pattern Recognition with Deep Spiking Neural Networks](https://mlsanity.com/paper/2007.01204/) (2020)
- [Recurrent Batch Normalization](https://mlsanity.com/paper/1603.09025/) (2016)
- [Whetstone: A Method for Training Deep Artificial Neural Networks for Binary Communication](https://mlsanity.com/paper/1810.11521/) (2018)
- [A learning framework for winner-take-all networks with stochastic synapses](https://mlsanity.com/paper/1708.04251/) (2017)
- [Recognizing Images with at most one Spike per Neuron](https://mlsanity.com/paper/2001.01682/) (2019)

## Built on by (most cited first, 10 of 10)

- [SpikingJelly: An open-source machine learning infrastructure platform for spike-based intelligence](https://mlsanity.com/paper/2310.16620/) (2023)
- [Spiking Neural Networks for event-based action recognition: A new task to understand their advantage](https://mlsanity.com/paper/2209.14915/) (2022)
- [Accurate and Efficient Event-based Semantic Segmentation Using Adaptive Spiking Encoder-Decoder Network](https://mlsanity.com/paper/2304.11857/) (2023)
- [Simple and complex spiking neurons: perspectives and analysis in a simple STDP scenario](https://mlsanity.com/paper/2207.04881/) (2022)
- [Hybrid Spiking Neural Network Fine-tuning for Hippocampus Segmentation](https://mlsanity.com/paper/2302.07328/) (2023)
- [Underwater Image Enhancement by Convolutional Spiking Neural Networks](https://mlsanity.com/paper/2503.20485/) (2025)
- [Parallel Hyperparameter Optimization Of Spiking Neural Network](https://mlsanity.com/paper/2403.00450/) (2024)
- [Fundamental Survey on Neuromorphic Based Audio Classification](https://mlsanity.com/paper/2502.15056/) (2025)
- [Frameworks for SNNs: a Review of Data Science-oriented Software and an Expansion of SpykeTorch](https://mlsanity.com/paper/2302.07624/) (2023)
- [DRiVE: Dynamic Recognition in VEhicles using snnTorch](https://mlsanity.com/paper/2502.10421/) (2025)

## Abstract (arXiv)

Spiking neural networks (SNNs) have become an interesting alternative to conventional artificial neural networks (ANN) thanks to their temporal processing capabilities and energy efficient implementations in neuromorphic hardware. However the challenges involved in training SNNs have limited their performance in terms of accuracy and thus their applications. Improving learning algorithms and neural architectures for a more accurate feature extraction is therefore one of the current priorities in SNN research. In this paper we present a study on the key components of modern spiking architectures. We design a spiking version of the successful residual network architecture and provide an in-depth study on the possible implementations of spiking residual connections. This study shows how, depending on the use case, the optimal residual connection implementation may vary. Additionally, we empirically compare different techniques in image classification datasets taken from the best performing networks. Our results provide a state of the art guide to SNN design, which allows to make informed choices when trying to build the optimal visual feature extractor. Finally, our network outperforms previous SNN architectures in CIFAR-10 (94.14%) and CIFAR-100 (74.65%) datasets and matches the state of the art in DVS-CIFAR10 (72.98%), with less parameters than the previous state of the art and without the need for ANN-SNN conversion. Code available at https://github.com/VicenteAlex/Spiking_ResNet