# Towards Low-latency Event-based Visual Recognition with Hybrid Step-wise Distillation Spiking Neural Networks

arXiv: https://arxiv.org/abs/2409.12507 · HTML: https://mlsanity.com/paper/2409.12507/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 43 references that are in the map.
- First posted on arXiv 19 Sep 2024.

## Builds on (most cited first, 21 of 21)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [FitNets: Hints for Thin Deep Nets](https://mlsanity.com/paper/1412.6550/) (2014)
- [Knowledge Distillation: A Survey](https://mlsanity.com/paper/2006.05525/) (2020)
- [Incorporating Learnable Membrane Time Constant to Enhance Learning of Spiking Neural Networks](https://mlsanity.com/paper/2007.05785/) (2020)
- [Converting Static Image Datasets to Spiking Neuromorphic Datasets Using Saccades](https://mlsanity.com/paper/1507.07629/) (2015)
- [Going Deeper With Directly-Trained Larger Spiking Neural Networks](https://mlsanity.com/paper/2011.05280/) (2020)
- [Spikformer: When Spiking Neural Network Meets Transformer](https://mlsanity.com/paper/2209.15425/) (2022)
- [Temporal Efficient Training of Spiking Neural Network via Gradient Re-weighting](https://mlsanity.com/paper/2202.11946/) (2022)
- [SpikingJelly: An open-source machine learning infrastructure platform for spike-based intelligence](https://mlsanity.com/paper/2310.16620/) (2023)
- [Optimal Conversion of Conventional Artificial Neural Networks to Spiking Neural Networks](https://mlsanity.com/paper/2103.00476/) (2021)
- [Temporal-wise Attention Spiking Neural Networks for Event Streams Classification](https://mlsanity.com/paper/2107.11711/) (2021)
- [Self-Supervised Learning of Event-Based Optical Flow with Spiking Neural Networks](https://mlsanity.com/paper/2106.01862/) (2021)
- [Optimized Potential Initialization for Low-latency Spiking Neural Networks](https://mlsanity.com/paper/2202.01440/) (2022)
- [Joint A-SNN: Joint Training of Artificial and Spiking Neural Networks via Self-Distillation and Weight Factorization](https://mlsanity.com/paper/2305.02099/) (2023)
- [Distilling Spikes: Knowledge Distillation in Spiking Neural Networks](https://mlsanity.com/paper/2005.00288/) (2020)
- [ES-ImageNet: A Million Event-Stream Classification Dataset for Spiking Neural Networks](https://mlsanity.com/paper/2110.12211/) (2021)
- [Shrinking Your TimeStep: Towards Low-Latency Neuromorphic Object Recognition with Spiking Neural Network](https://mlsanity.com/paper/2401.01912/) (2024)
- [Unleashing the Potential of Spiking Neural Networks by Dynamic Confidence](https://mlsanity.com/paper/2303.10276/) (2023)
- [Input-Aware Dynamic Timestep Spiking Neural Networks for Efficient In-Memory Computing](https://mlsanity.com/paper/2305.17346/) (2023)
- [An Efficient Knowledge Transfer Strategy for Spiking Neural Networks from Static to Event Domain](https://mlsanity.com/paper/2303.13077/) (2023)

## Built on by (most cited first, 8 of 8)

- [SU-YOLO: Spiking Neural Network for Efficient Underwater Object Detection](https://mlsanity.com/paper/2503.24389/) (2025)
- [SOTA: Spike-Navigated Optimal TrAnsport Saliency Region Detection in Composite-bias Videos](https://mlsanity.com/paper/2505.00394/) (2025)
- [Edge Intelligence with Spiking Neural Networks](https://mlsanity.com/paper/2507.14069/) (2025)
- [SpikeDerain: Unveiling Clear Videos from Rainy Sequences Using Color Spike Streams](https://mlsanity.com/paper/2503.20315/) (2025)
- [Brain-Inspired Multimodal Spiking Neural Network for Image-Text Retrieval](https://mlsanity.com/paper/2603.26787/) (2026)
- [A2SG:Adaptive and Asymmetric Surrogate Gradients for Training Deep Spiking Neural Networks](https://mlsanity.com/paper/2606.11236/) (2026)
- [Adaptive Time-step Training for Enhancing Spike-Based Neural Radiance Fields](https://mlsanity.com/paper/2507.23033/) (2025)
- [A Robust Point Cloud Analysis Framework Inspired By Primary Visual Cortex](https://mlsanity.com/paper/2606.14292/) (2026)

## Abstract (arXiv)

Spiking neural networks (SNNs) have garnered significant attention for their low power consumption and high biological interpretability. Their rich spatio-temporal information processing capability and event-driven nature make them ideally well-suited for neuromorphic datasets. However, current SNNs struggle to balance accuracy and latency in classifying these datasets. In this paper, we propose Hybrid Step-wise Distillation (HSD) method, tailored for neuromorphic datasets, to mitigate the notable decline in performance at lower time steps. Our work disentangles the dependency between the number of event frames and the time steps of SNNs, utilizing more event frames during the training stage to improve performance, while using fewer event frames during the inference stage to reduce latency. Nevertheless, the average output of SNNs across all time steps is susceptible to individual time step with abnormal outputs, particularly at extremely low time steps. To tackle this issue, we implement Step-wise Knowledge Distillation (SKD) module that considers variations in the output distribution of SNNs at each time step. Empirical evidence demonstrates that our method yields competitive performance in classification tasks on neuromorphic datasets, especially at lower time steps. Our code will be available at: {https://github.com/hsw0929/HSD}.