# Spatio-Temporal Pruning and Quantization for Low-latency Spiking Neural Networks

arXiv: https://arxiv.org/abs/2104.12528 · HTML: https://mlsanity.com/paper/2104.12528/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 41 references that are in the map.
- First posted on arXiv 26 Apr 2021.

## Builds on (most cited first, 24 of 24)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding](https://mlsanity.com/paper/1510.00149/) (2015)
- [Learning both Weights and Connections for Efficient Neural Networks](https://mlsanity.com/paper/1506.02626/) (2015)
- [Surrogate Gradient Learning in Spiking Neural Networks](https://mlsanity.com/paper/1901.09948/) (2019)
- [Learning Structured Sparsity in Deep Neural Networks](https://mlsanity.com/paper/1608.03665/) (2016)
- [HAQ: Hardware-Aware Automated Quantization with Mixed Precision](https://mlsanity.com/paper/1811.08886/) (2018)
- [Spatio-Temporal Backpropagation for Training High-performance Spiking Neural Networks](https://mlsanity.com/paper/1706.02609/) (2017)
- [On the Efficacy of Knowledge Distillation](https://mlsanity.com/paper/1910.01348/) (2019)
- [Tensorizing Neural Networks](https://mlsanity.com/paper/1509.06569/) (2015)
- [Going Deeper in Spiking Neural Networks: VGG and Residual Architectures](https://mlsanity.com/paper/1802.02627/) (2018)
- [Enabling Deep Spiking Neural Networks with Hybrid Conversion and Spike Timing Dependent Backpropagation](https://mlsanity.com/paper/2005.01807/) (2020)
- [Convolutional Networks for Fast, Energy-Efficient Neuromorphic Computing](https://mlsanity.com/paper/1603.08270/) (2016)
- [Spiking Deep Networks with LIF Neurons](https://mlsanity.com/paper/1510.08829/) (2015)
- [Inherent Adversarial Robustness of Deep Spiking Neural Networks: Effects of Discrete Input Encoding and Non-Linear Activations](https://mlsanity.com/paper/2003.10399/) (2020)
- [STDP Based Pruning of Connections and Weight Quantization in Spiking Neural Networks for Energy Efficient Recognition](https://mlsanity.com/paper/1710.04734/) (2017)
- [T2FSNN: Deep Spiking Neural Networks with Time-to-first-spike Coding](https://mlsanity.com/paper/2003.11741/) (2020)
- [Comprehensive SNN Compression Using ADMM Optimization and Activity Regularization](https://mlsanity.com/paper/1911.00822/) (2019)
- [Exploring the Connection Between Binary and Spiking Neural Networks](https://mlsanity.com/paper/2002.10064/) (2020)
- [Fast and Efficient Information Transmission with Burst Spikes in Deep Spiking Neural Networks](https://mlsanity.com/paper/1809.03142/) (2018)
- [Distilling Spikes: Knowledge Distillation in Spiking Neural Networks](https://mlsanity.com/paper/2005.00288/) (2020)
- [Towards Understanding the Effect of Leak in Spiking Neural Networks](https://mlsanity.com/paper/2006.08761/) (2020)
- [A Low Effort Approach to Structured CNN Design Using PCA](https://mlsanity.com/paper/1812.06224/) (2018)
- [ReStoCNet: Residual Stochastic Binary Convolutional Spiking Neural Network for Memory-Efficient Neuromorphic Computing](https://mlsanity.com/paper/1902.04161/) (2019)
- [DCT-SNN: Using DCT to Distribute Spatial Information over Time for Learning Low-Latency Spiking Neural Networks](https://mlsanity.com/paper/2010.01795/) (2020)

## Built on by (most cited first, 21 of 21)

- [Direct Learning-Based Deep Spiking Neural Networks: A Review](https://mlsanity.com/paper/2305.19725/) (2023)
- [Exploring Temporal Information Dynamics in Spiking Neural Networks](https://mlsanity.com/paper/2211.14406/) (2022)
- [Energy-efficient Knowledge Distillation for Spiking Neural Networks](https://mlsanity.com/paper/2106.07172/) (2021)
- [Input-Aware Dynamic Timestep Spiking Neural Networks for Efficient In-Memory Computing](https://mlsanity.com/paper/2305.17346/) (2023)
- [MINT: Multiplier-less INTeger Quantization for Energy Efficient Spiking Neural Networks](https://mlsanity.com/paper/2305.09850/) (2023)
- [Sparse Spiking Neural Network: Exploiting Heterogeneity in Timescales for Pruning Recurrent SNN](https://mlsanity.com/paper/2403.03409/) (2024)
- [Towards Efficient Deep Spiking Neural Networks Construction with Spiking Activity based Pruning](https://mlsanity.com/paper/2406.01072/) (2024)
- [Ternary Spike-based Neuromorphic Signal Processing System](https://mlsanity.com/paper/2407.05310/) (2024)
- [Resource Constrained Model Compression via Minimax Optimization for Spiking Neural Networks](https://mlsanity.com/paper/2308.04672/) (2023)
- [SQUAT: Stateful Quantization-Aware Training in Recurrent Spiking Neural Networks](https://mlsanity.com/paper/2404.19668/) (2024)
- [Memory-Free and Parallel Computation for Quantized Spiking Neural Networks](https://mlsanity.com/paper/2503.00040/) (2025)
- [Spatio-Temporal Pruning for Compressed Spiking Large Language Models](https://mlsanity.com/paper/2508.20122/) (2025)
- [Energy-Efficient and Dequantization-Free Q-LLMs: A Spiking Neural Network Approach to Salient Value Mitigation](https://mlsanity.com/paper/2510.19498/) (2025)
- [The Hardware Impact of Quantization and Pruning for Weights in Spiking Neural Networks](https://mlsanity.com/paper/2302.04174/) (2023)
- [Toward Efficient Deep Spiking Neuron Networks:A Survey On Compression](https://mlsanity.com/paper/2407.08744/) (2024)
- [Synaptic Modulation using Interspike Intervals Increases Energy Efficiency of Spiking Neural Networks](https://mlsanity.com/paper/2408.02961/) (2024)
- [Spiking Brain Compression: Post-Training Second-order Compression for Spiking Neural Networks](https://mlsanity.com/paper/2506.03996/) (2025)
- [SPEAR: Structured Pruning for Spiking Neural Networks via Synaptic Operation Estimation and Reinforcement Learning](https://mlsanity.com/paper/2507.02945/) (2025)
- [Incorporating the Refractory Period into Spiking Neural Networks through Spike-Triggered Threshold Dynamics](https://mlsanity.com/paper/2509.17769/) (2025)
- [MD-SNN: Membrane Potential-aware Distillation on Quantized Spiking Neural Network](https://mlsanity.com/paper/2512.04443/) (2025)
- [BASC : Behavior-Aligned Quantization and Pruning for Low-Bit Spiking Neural Networks](https://mlsanity.com/paper/2608.19239/) (2026)

## Abstract (arXiv)

Spiking Neural Networks (SNNs) are a promising alternative to traditional deep learning methods since they perform event-driven information processing. However, a major drawback of SNNs is high inference latency. The efficiency of SNNs could be enhanced using compression methods such as pruning and quantization. Notably, SNNs, unlike their non-spiking counterparts, consist of a temporal dimension, the compression of which can lead to latency reduction. In this paper, we propose spatial and temporal pruning of SNNs. First, structured spatial pruning is performed by determining the layer-wise significant dimensions using principal component analysis of the average accumulated membrane potential of the neurons. This step leads to 10-14X model compression. Additionally, it enables inference with lower latency and decreases the spike count per inference. To further reduce latency, temporal pruning is performed by gradually reducing the timesteps while training. The networks are trained using surrogate gradient descent based backpropagation and we validate the results on CIFAR10 and CIFAR100, using VGG architectures. The spatiotemporally pruned SNNs achieve 89.04% and 66.4% accuracy on CIFAR10 and CIFAR100, respectively, while performing inference with 3-30X reduced latency compared to state-of-the-art SNNs. Moreover, they require 8-14X lesser compute energy compared to their unpruned standard deep learning counterparts. The energy numbers are obtained by multiplying the number of operations with energy per operation. These SNNs also provide 1-4% higher robustness against Gaussian noise corrupted inputs. Furthermore, we perform weight quantization and find that performance remains reasonably stable up to 5-bit quantization.