# Surrogates, Spikes, and Sparsity: Performance Analysis and Characterization of SNN Hyperparameters on Hardware

arXiv: https://arxiv.org/abs/2603.24891 · HTML: https://mlsanity.com/paper/2603.24891/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 34 references that are in the map.
- First posted on arXiv 26 Mar 2026.

## Builds on (most cited first, 16 of 16)

- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [Optuna: A Next-generation Hyperparameter Optimization Framework](https://mlsanity.com/paper/1907.10902/) (2019)
- [Surrogate Gradient Learning in Spiking Neural Networks](https://mlsanity.com/paper/1901.09948/) (2019)
- [SLAYER: Spike Layer Error Reassignment in Time](https://mlsanity.com/paper/1810.08646/) (2018)
- [Training Spiking Neural Networks Using Lessons From Deep Learning](https://mlsanity.com/paper/2109.12894/) (2021)
- [SATA: Sparsity-Aware Training Accelerator for Spiking Neural Networks](https://mlsanity.com/paper/2204.05422/) (2022)
- [Spiking Neural Networks Hardware Implementations and Challenges: a Survey](https://mlsanity.com/paper/2005.01467/) (2020)
- [Workload-Balanced Pruning for Sparse Spiking Neural Networks](https://mlsanity.com/paper/2302.06746/) (2023)
- [LoAS: Fully Temporal-Parallel Dataflow for Dual-Sparse Spiking Neural Networks](https://mlsanity.com/paper/2407.14073/) (2024)
- [Prosperity: Accelerating Spiking Neural Networks via Product Sparsity](https://mlsanity.com/paper/2503.03379/) (2025)
- [Efficient Hardware Acceleration of Sparsely Active Convolutional Spiking Neural Networks](https://mlsanity.com/paper/2203.12437/) (2022)
- [SNE: an Energy-Proportional Digital Accelerator for Sparse Event-Based Convolutions](https://mlsanity.com/paper/2204.10687/) (2022)
- [Fine-Tuning Surrogate Gradient Learning for Optimal Hardware Performance in Spiking Neural Networks](https://mlsanity.com/paper/2402.06211/) (2024)
- [FireFly-S: Exploiting Dual-Side Sparsity for Spiking Neural Networks Acceleration with Reconfigurable Spatial Architecture](https://mlsanity.com/paper/2408.15578/) (2024)
- [PULSE: Parametric Hardware Units for Low-power Sparsity-Aware Convolution Engine](https://mlsanity.com/paper/2402.06210/) (2024)

## Abstract (arXiv)

Spiking Neural Networks (SNNs) offer inherent advantages for low-power inference through sparse, event-driven computation. However, the theoretical energy benefits of SNNs are often decoupled from real hardware performance due to the opaque relationship between training-time choices and inference-time sparsity. While prior work has focused on weight pruning and compression, the role of training hyperparameters -- specifically surrogate gradient functions and neuron model configurations -- in shaping hardware-level activation sparsity remains underexplored. This paper presents a workload characterization study quantifying the sensitivity of hardware latency to SNN hyperparameters. We decouple the impact of surrogate gradient functions (e.g., Fast Sigmoid, Spike Rate Escape) and neuron models (LIF, Lapicque) on classification accuracy and inference efficiency across three event-based vision datasets: DVS128-Gesture, N-MNIST, and DVS-CIFAR10. Our analysis reveals that standard accuracy metrics are poor predictors of hardware efficiency. While Fast Sigmoid achieves the highest accuracy on DVS-CIFAR10, Spike Rate Escape reduces inference latency by up to 12.2% on DVS128-Gesture with minimal accuracy trade-offs. We also demonstrate that neuron model selection is as critical as parameter tuning; transitioning from LIF to Lapicque neurons yields up to 28% latency reduction. We validate on a custom cycle-accurate FPGA-based SNN instrumentation platform, showing that sparsity-aware hyperparameter selection can improve accuracy by 9.1% and latency by over 2x compared to baselines. These findings establish a methodology for predicting hardware behavior from training parameters. The RTL and reproducibility artifacts are at https://zenodo.org/records/18893738.