mlsanityData to 14 Oct 2026

Distillation for spiking neural networks

23 papers0 this weekHigh confidence in this grouping: the papers form one clear topicHigh confidence in this grouping: the papers form one clear topic · About

Keywords: snns, spiking, snn, neural, spiking neural

Key facts. Distillation for spiking neural networks: 23 papers on arXiv since 2021 (data through 14 Oct 2026). 11 of its 23 mapped papers (48%) cite “Constructing Deep Spiking Neural Networks from Artificial Neural Networks with Knowledge Distillation” (2023, arXiv 2304.05627).

What’s new

Nothing new since 25 Sep. Latest: 28 Aug 2026.

Foundations

Most cited by papers here

  1. Constructing Deep Spiking Neural Networks from Artificial Neural Networks with Knowledge Distillation2023cited by 11 here
  2. Joint A-SNN: Joint Training of Artificial and Spiking Neural Networks via Self-Distillation and Weight Factorization2023cited by 8 here
  3. BKDSNN: Enhancing the Performance of Learning-based Spiking Neural Networks Training with Blurred Knowledge Distillation2024cited by 7 here
  4. Energy-efficient Knowledge Distillation for Spiking Neural Networks2021cited by 5 here
  5. Biologically inspired structure learning with reverse knowledge distillation for spiking neural networks2023cited by 4 here

Lineage

The papers this direction builds on, oldest first.

Each paper on the line cites the one before.

  1. 2021Focal and Global Knowledge Distillation for Detectorscited by 1 here
  2. 2022Task-Balanced Distillation for Object Detectioncited by 1 here
  3. 2023Joint A-SNN: Joint Training of Artificial and Spiking Neural Networks via Self-Distillation and Weight Factorizationcited by 8 here
  4. 2024Self-Distillation Learning Based on Temporal-Spatial Consistency for Spiking Neural Networkscited by 3 here
20 more it builds on, not in this chain
  1. 2021Rethinking Soft Labels for Knowledge Distillation: A Bias-Variance Tradeoff Perspectivecited by 1 here
  2. 2021Refine Myself by Teaching Myself: Feature Refinement via Self-Knowledge Distillationcited by 1 here
  3. 2021Distilling Knowledge via Knowledge Reviewcited by 2 here
  4. 2021Contrastive Model Inversion for Data-Free Knowledge Distillationcited by 1 here
  5. 2021Comparing Kullback-Leibler Divergence and Mean Squared Error Loss in Knowledge Distillationcited by 1 here
  6. 2021Energy-efficient Knowledge Distillation for Spiking Neural Networkscited by 5 here
  7. 2022Robust and Resource-Efficient Data-Free Knowledge Distillation by Generative Pseudo Replaycited by 1 here
  8. 2022Decoupled Knowledge Distillationcited by 2 here
  9. 2022Knowledge Distillation from A Stronger Teachercited by 2 here
  10. 2022Better Teacher Better Student: Dynamic Prior Knowledge for Knowledge Distillationcited by 2 here
  11. 2022Efficient One Pass Self-distillation with Zipf's Label Smoothingcited by 1 here
  12. 2023From Knowledge Distillation to Self-Knowledge Distillation: A Unified Approach with Normalized Loss and Customized Soft Labelscited by 3 here
  13. 2023Constructing Deep Spiking Neural Networks from Artificial Neural Networks with Knowledge Distillationcited by 11 here
  14. 2023Biologically inspired structure learning with reverse knowledge distillation for spiking neural networkscited by 4 here
  15. 2023NORM: Knowledge Distillation via N-to-One Representation Matchingcited by 1 here
  16. 2023Learning to Learn from APIs: Black-Box Data-Free Meta-Learningcited by 1 here
  17. 2023SpikeBERT: A Language Spikformer Learned from BERT with Knowledge Distillationcited by 2 here
  18. 2024Logit Standardization in Knowledge Distillationcited by 2 here
  19. 2024BKDSNN: Enhancing the Performance of Learning-based Spiking Neural Networks Training with Blurred Knowledge Distillationcited by 7 here
  20. 2025Temporal Separation with Entropy Regularization for Knowledge Distillation in Spiking Neural Networkscited by 3 here

The 24 most cited of 35.

Browse

18 of 23, ranked 6–23

  1. 6Temporal Separation with Entropy Regularization for Knowledge Distillation in Spiking Neural Networks20253
  2. 7Self-Distillation Learning Based on Temporal-Spatial Consistency for Spiking Neural Networks20243
  3. 8SpikeBERT: A Language Spikformer Learned from BERT with Knowledge Distillation20232
  4. 9Efficient Logit-based Knowledge Distillation of Deep Spiking Neural Networks for Full-Range Timestep Deployment20251
  5. 10Head-Tail-Aware KL Divergence in Knowledge Distillation for Spiking Neural Networks20251
  6. 11Cross Knowledge Distillation between Artificial and Spiking Neural Networks20251
  7. 12Synergy Between the Strong and the Weak: Spiking Neural Networks are Inherently Self-Distillers20251
  8. 13BLEND: Behavior-guided Neural Population Dynamics Modeling via Privileged Knowledge Distillation20241
  9. 14STARS: Spike Tail-Aware Relational Synthesis for ANN-to-SNN Data-Free Knowledge Distillation20260
  10. 15Not All Timesteps Matter Equally: Selective Alignment Knowledge Distillation for Spiking Neural Networks20260
All 18
  1. 16SpikeOPD: Stable On-Policy Distillation for Autoregressive Spiking Language Models20260
  2. 17Dynamic Activation with Knowledge Distillation for Energy-Efficient Spiking NN Ensembles20250
  3. 18Enhanced Self-Distillation Framework for Efficient Spiking Neural Network Training20250
  4. 19SpikingMamba: Towards Energy-Efficient Large Language Models via Knowledge Distillation from Mamba20250
  5. 20A Closer Look at Knowledge Distillation in Spiking Neural Network Training20250
  6. 21Cross-Modal Representational Knowledge Distillation for Enhanced Spike-Informed LFP Modeling20250
  7. 22Spiking CenterNet: A Distillation-boosted Spiking Neural Network for Object Detection20240
  8. 23Efficient Speech Command Recognition Leveraging Spiking Neural Network and Curriculum Learning-based Knowledge Distillation20240