# Not All Timesteps Matter Equally: Selective Alignment Knowledge Distillation for Spiking Neural Networks

arXiv: https://arxiv.org/abs/2605.14252 · HTML: https://mlsanity.com/paper/2605.14252/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Distillation for spiking neural networks, one of 12 directions in Knowledge distillation.
- Builds on 23 of its 32 references that are in the map.
- First posted on arXiv 14 May 2026.

## Placed in

- Knowledge distillation / [Distillation for spiking neural networks](https://mlsanity.com/distill/distillation-for-spiking-neural-networks/)

## Builds on (most cited first, 23 of 23)

- [Learning both Weights and Connections for Efficient Neural Networks](https://mlsanity.com/paper/1506.02626/) (2015)
- [Surrogate Gradient Learning in Spiking Neural Networks](https://mlsanity.com/paper/1901.09948/) (2019)
- [Converting Static Image Datasets to Spiking Neuromorphic Datasets Using Saccades](https://mlsanity.com/paper/1507.07629/) (2015)
- [Going Deeper With Directly-Trained Larger Spiking Neural Networks](https://mlsanity.com/paper/2011.05280/) (2020)
- [Spikformer: When Spiking Neural Network Meets Transformer](https://mlsanity.com/paper/2209.15425/) (2022)
- [Temporal Efficient Training of Spiking Neural Network via Gradient Re-weighting](https://mlsanity.com/paper/2202.11946/) (2022)
- [Long short-term memory and learning-to-learn in networks of spiking neurons](https://mlsanity.com/paper/1803.09574/) (2018)
- [Spike-driven Transformer](https://mlsanity.com/paper/2307.01694/) (2023)
- [GLIF: A Unified Gated Leaky Integrate-and-Fire Neuron for Spiking Neural Networks](https://mlsanity.com/paper/2210.13768/) (2022)
- [Constructing Deep Spiking Neural Networks from Artificial Neural Networks with Knowledge Distillation](https://mlsanity.com/paper/2304.05627/) (2023)
- [Towards Memory- and Time-Efficient Backpropagation for Training Spiking Neural Networks](https://mlsanity.com/paper/2302.14311/) (2023)
- [Joint A-SNN: Joint Training of Artificial and Spiking Neural Networks via Self-Distillation and Weight Factorization](https://mlsanity.com/paper/2305.02099/) (2023)
- [LaSNN: Layer-wise ANN-to-SNN Distillation for Effective and Efficient Training in Deep Spiking Neural Networks](https://mlsanity.com/paper/2304.09101/) (2023)
- [Rethinking Spiking Neural Networks from an Ensemble Learning Perspective](https://mlsanity.com/paper/2502.14218/) (2025)
- [BKDSNN: Enhancing the Performance of Learning-based Spiking Neural Networks Training with Blurred Knowledge Distillation](https://mlsanity.com/paper/2407.09083/) (2024)
- [Improving Stability and Performance of Spiking Neural Networks through Enhancing Temporal Consistency](https://mlsanity.com/paper/2305.14174/) (2023)
- [Temporal Separation with Entropy Regularization for Knowledge Distillation in Spiking Neural Networks](https://mlsanity.com/paper/2503.03144/) (2025)
- [Self-Distillation Learning Based on Temporal-Spatial Consistency for Spiking Neural Networks](https://mlsanity.com/paper/2406.07862/) (2024)
- [Advancing Training Efficiency of Deep Spiking Neural Networks through Rate-based Backpropagation](https://mlsanity.com/paper/2410.11488/) (2024)
- [Efficient ANN-Guided Distillation: Aligning Rate-based Features of Spiking Neural Networks through Hybrid Block-wise Replacement](https://mlsanity.com/paper/2503.16572/) (2025)
- [Efficient Logit-based Knowledge Distillation of Deep Spiking Neural Networks for Full-Range Timestep Deployment](https://mlsanity.com/paper/2501.15925/) (2025)
- [Synergy Between the Strong and the Weak: Spiking Neural Networks are Inherently Self-Distillers](https://mlsanity.com/paper/2510.07924/) (2025)
- [Head-Tail-Aware KL Divergence in Knowledge Distillation for Spiking Neural Networks](https://mlsanity.com/paper/2504.20445/) (2025)

## Abstract (arXiv)

Spiking neural networks (SNNs), which are brain-inspired and spike-driven, achieve high energy efficiency. However, a performance gap between SNNs and artificial neural networks (ANNs) still remains. Knowledge distillation (KD) is commonly adopted to improve SNN performance, but existing methods typically enforce uniform alignment across all timesteps, either from a teacher network or through inter-temporal self-distillation, implicitly assuming that per-timestep predictions should be treated equally. In practice, SNN predictions vary and evolve over time, and intermediate timesteps need not all be individually correct even when the final aggregated output is correct. Under such conditions, effective distillation should not force every timestep toward the same supervision target, but instead provide corrective guidance to erroneous timesteps while preserving useful temporal dynamics. To address this issue, we propose Selective Alignment Knowledge Distillation (SeAl-KD), which selectively aligns class-level and temporal knowledge by equalizing competing logits at erroneous timesteps and reweighting temporal alignment based on confidence and inter-timestep similarity. Extensive experiments on static image and neuromorphic event-based datasets demonstrate consistent improvements over existing distillation methods. The code is available at https://github.com/KaiSUN1/SeAl