mlsanityData to 7 Oct 2026

Biologically inspired structure learning with reverse knowledge distillation for spiking neural networks

arXiv 2304.09500 ↗First posted 19 Apr 2023cs.NE cs.AIcited by 11

Not on the map. Why?

Abstract (arXiv)

Spiking neural networks (SNNs) have superb characteristics in sensory information recognition tasks due to their biological plausibility. However, the performance of some current spiking-based models is limited by their structures which means either fully connected or too-deep structures bring too much redundancy. This redundancy from both connection and neurons is one of the key factors hindering the practical application of SNNs. Although Some pruning methods were proposed to tackle this problem, they normally ignored the fact the neural topology in the human brain could be adjusted dynamically. Inspired by this, this paper proposed an evolutionary-based structure construction method for constructing more reasonable SNNs. By integrating the knowledge distillation and connection pruning method, the synaptic connections in SNNs can be optimized dynamically to reach an optimal state. As a result, the structure of SNNs could not only absorb knowledge from the teacher model but also search for deep but sparse network topology. Experimental results on CIFAR100 and DVS-Gesture show that the proposed structure learning method can get pretty well performance while reducing the connection redundancy. The proposed method explores a novel dynamical way for structure learning from scratch in SNNs which could build a bridge to close the gap between deep learning and bio-inspired neural dynamics.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 18 of its 30 references that are in the map. First posted on arXiv 19 Apr 2023.

Builds on

18 of 30 references.

References · 18

  1. 2019Surrogate Gradient Learning in Spiking Neural Networkscited by 604
  2. 2017Spatio-Temporal Backpropagation for Training High-performance Spiking Neural Networkscited by 399
  3. 2020Incorporating Learnable Membrane Time Constant to Enhance Learning of Spiking Neural Networkscited by 340
  4. 2020Going Deeper With Directly-Trained Larger Spiking Neural Networkscited by 268
  5. 2018SLAYER: Spike Layer Error Reassignment in Timecited by 257
  6. 2022Temporal Efficient Training of Spiking Neural Network via Gradient Re-weightingcited by 192
  7. 2019Spiking-YOLO: Spiking Neural Network for Energy-Efficient Object Detectioncited by 174
  8. 2020Enabling Deep Spiking Neural Networks with Hybrid Conversion and Spike Timing Dependent Backpropagationcited by 142
  9. 2020RMP-SNN: Residual Membrane Potential Neuron for Enabling Deeper High-Accuracy and Low-Latency Spiking Neural Networkcited by 141
  10. 2021Optimal Conversion of Conventional Artificial Neural Networks to Spiking Neural Networkscited by 134
  11. 2023Constructing Deep Spiking Neural Networks from Artificial Neural Networks with Knowledge Distillationcited by 41
  12. 2020Comparing SNNs and RNNs on Neuromorphic Vision Datasets: Similarities and Differencescited by 39
6 more
  1. 2018Fast and Efficient Information Transmission with Burst Spikes in Deep Spiking Neural Networkscited by 34
  2. 2020Distilling Spikes: Knowledge Distillation in Spiking Neural Networkscited by 24
  3. 2022Real Spike: Learning Real-valued Spikes for Spiking Neural Networkscited by 22
  4. 2020TCL: an ANN-to-SNN Conversion with Trainable Clipping Layerscited by 20
  5. 2021Energy-efficient Knowledge Distillation for Spiking Neural Networkscited by 10
  6. 2019On-chip Few-shot Learning with Surrogate Gradient Descent on a Neuromorphic Processorcited by 8

Built on by

Cited by year

Citing papers per year: 2023: 3; 2024: 3; 2025: 5.

11 papers.

  1. 2023Membrane Potential Batch Normalization for Spiking Neural Networkscited by 35
  2. 2023Direct Learning-Based Deep Spiking Neural Networks: A Reviewcited by 31
  3. 2023RMP-Loss: Regularizing Membrane Potential Distribution for Spiking Neural Networkscited by 27
  4. 2025STAA-SNN: Spatial-Temporal Attention Aggregator for Spiking Neural Networkscited by 11
  5. 2025Temporal Separation with Entropy Regularization for Knowledge Distillation in Spiking Neural Networkscited by 10
  6. 2024Self-Distillation Learning Based on Temporal-Spatial Consistency for Spiking Neural Networkscited by 9
  7. 2024Spiking CenterNet: A Distillation-boosted Spiking Neural Network for Object Detectioncited by 8
  8. 2025TS-SNN: Temporal Shift Module for Spiking Neural Networkscited by 5
  9. 2024Apprenticeship-Inspired Elegance: Synergistic Knowledge Distillation Empowers Spiking Neural Networks for Efficient Single-Eye Emotion Recognitioncited by 3
  10. 2025Efficient Logit-based Knowledge Distillation of Deep Spiking Neural Networks for Full-Range Timestep Deploymentcited by 2
  11. 2025Optimization of Low-Latency Spiking Neural Networks Utilizing Historical Dynamics of Refractory Periodscited by 0