# Hessian Aware Quantization of Spiking Neural Networks

arXiv: https://arxiv.org/abs/2104.14117 · HTML: https://mlsanity.com/paper/2104.14117/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 19 references that are in the map.
- First posted on arXiv 29 Apr 2021.

## Builds on (most cited first, 7 of 7)

- [Learned Step Size Quantization](https://mlsanity.com/paper/1902.08153/) (2019)
- [Converting Static Image Datasets to Spiking Neuromorphic Datasets Using Saccades](https://mlsanity.com/paper/1507.07629/) (2015)
- [HAWQ-V2: Hessian Aware trace-Weighted Quantization of Neural Networks](https://mlsanity.com/paper/1911.03852/) (2019)
- [Convolutional Networks for Fast, Energy-Efficient Neuromorphic Computing](https://mlsanity.com/paper/1603.08270/) (2016)
- [Brain-Inspired Learning on Neuromorphic Substrates](https://mlsanity.com/paper/2010.11931/) (2020)
- [QPyTorch: A Low-Precision Arithmetic Simulation Framework](https://mlsanity.com/paper/1910.04540/) (2019)
- [Rounding Methods for Neural Networks with Low Resolution Synaptic Weights](https://mlsanity.com/paper/1504.05767/) (2015)

## Built on by (most cited first, 7 of 7)

- [MINT: Multiplier-less INTeger Quantization for Energy Efficient Spiking Neural Networks](https://mlsanity.com/paper/2305.09850/) (2023)
- [Low Precision Quantization-aware Training in Spiking Neural Networks with Differentiable Quantization Function](https://mlsanity.com/paper/2305.19295/) (2023)
- [SQUAT: Stateful Quantization-Aware Training in Recurrent Spiking Neural Networks](https://mlsanity.com/paper/2404.19668/) (2024)
- [SpikeFit: Towards Optimal Deployment of Spiking Networks on Neuromorphic Hardware](https://mlsanity.com/paper/2510.15542/) (2025)
- [The Hardware Impact of Quantization and Pruning for Weights in Spiking Neural Networks](https://mlsanity.com/paper/2302.04174/) (2023)
- [Spiking Brain Compression: Post-Training Second-order Compression for Spiking Neural Networks](https://mlsanity.com/paper/2506.03996/) (2025)
- [PTQ4SNN: Membrane-Aware Post-Training Quantization for Spiking Neural Networks](https://mlsanity.com/paper/2608.07066/) (2026)

## Abstract (arXiv)

To achieve the low latency, high throughput, and energy efficiency benefits of Spiking Neural Networks (SNNs), reducing the memory and compute requirements when running on a neuromorphic hardware is an important step. Neuromorphic architecture allows massively parallel computation with variable and local bit-precisions. However, how different bit-precisions should be allocated to different layers or connections of the network is not trivial. In this work, we demonstrate how a layer-wise Hessian trace analysis can measure the sensitivity of the loss to any perturbation of the layer's weights, and this can be used to guide the allocation of a layer-specific bit-precision when quantizing an SNN. In addition, current gradient based methods of SNN training use a complex neuron model with multiple state variables, which is not ideal for compute and memory efficiency. To address this challenge, we present a simplified neuron model that reduces the number of state variables by 4-fold while still being compatible with gradient based training. We find that the impact on model accuracy when using a layer-wise bit-precision correlated well with that layer's Hessian trace. The accuracy of the optimal quantized network only dropped by 0.2%, yet the network size was reduced by 58%. This reduces memory usage and allows fixed-point arithmetic with simpler digital circuits to be used, increasing the overall throughput and energy efficiency.