mlsanityData to 7 Oct 2026

STARS: Spike Tail-Aware Relational Synthesis for ANN-to-SNN Data-Free Knowledge Distillation

arXiv 2605.27409 ↗First posted 12 May 2026cs.NE cs.AI cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

SNNs promise energy-efficient and low-latency inference, but their performance still trails that of ANNs. ANN-to-SNN knowledge distillation helps narrow this gap, yet the original training data are often unavailable in practical deployment settings. Existing data-free knowledge distillation (DFKD) methods synthesize surrogate data by matching teacher-side priors, especially BN statistics, but these ANN-oriented constraints mainly regularize mean and variance and therefore remain under-constrained for SNN students whose responses depend on threshold-crossing dynamics. In this paper, we propose Spike Tail-Aware Relational Synthesis (STARS), a plug-and-play method for ANN-to-SNN DFKD that augments standard BN-guided synthesis with two complementary objectives: Relational Consistency Alignment, which preserves cross-sample relational consistency between teacher and student, and Tail-Aware Regularization, which regularizes threshold-relevant tail probabilities through soft exceedance over teacher-derived thresholds. Together, these objectives generate synthetic batches that remain teacher-valid while becoming more informative for SNN students. Experiments on CIFAR-10, CIFAR-100, and Tiny-ImageNet across multiple ANN-SNN pairs show that our method consistently improves conventional DFKD baselines and even surpasses several KD methods, with gains of up to 4.6% on CIFAR-10 and 6.7% on CIFAR-100, highlighting the importance of complementing BN matching with relational and tail-aware constraints in SNN-oriented DFKD.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 36 of its 49 references that are in the map. First posted on arXiv 12 May 2026.

Builds on

36 of 49 references.

References · 36

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  4. 2014FitNets: Hints for Thin Deep Netscited by 1,619
  5. 2016Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfercited by 1,059
  6. 2019Relational Knowledge Distillationcited by 873
  7. 2019Contrastive Representation Distillationcited by 734
  8. 2019Surrogate Gradient Learning in Spiking Neural Networkscited by 604
  9. 2019Dreaming to Distill: Data-free Knowledge Transfer via DeepInversioncited by 419
  10. 2017Spatio-Temporal Backpropagation for Training High-performance Spiking Neural Networkscited by 399
  11. 2020Incorporating Learnable Membrane Time Constant to Enhance Learning of Spiking Neural Networkscited by 340
  12. 2020Going Deeper With Directly-Trained Larger Spiking Neural Networkscited by 268
24 more
  1. 2022Spikformer: When Spiking Neural Network Meets Transformercited by 263
  2. 2019Data-Free Learning of Student Networkscited by 203
  3. 2021Optimal Conversion of Conventional Artificial Neural Networks to Spiking Neural Networkscited by 134
  4. 2019Zero-shot Knowledge Transfer via Adversarial Belief Matchingcited by 129
  5. 2019Zero-Shot Knowledge Distillation in Deep Networkscited by 119
  6. 2019Data-Free Adversarial Distillationcited by 90
  7. 2021Contrastive Model Inversion for Data-Free Knowledge Distillationcited by 66
  8. 2022GLIF: A Unified Gated Leaky Integrate-and-Fire Neuron for Spiking Neural Networkscited by 63
  9. 2020Progressive Tandem Learning for Pattern Recognition with Deep Spiking Neural Networkscited by 46
  10. 2022Robust and Resource-Efficient Data-Free Knowledge Distillation by Generative Pseudo Replaycited by 35
  11. 2023Joint A-SNN: Joint Training of Artificial and Spiking Neural Networks via Self-Distillation and Weight Factorizationcited by 34
  12. 2022Training Spiking Neural Networks with Local Tandem Learningcited by 22
  13. 2023NAYER: Noisy Layer Data Generation for Efficient and Effective Data-free Knowledge Distillationcited by 22
  14. 2023Learning to Learn from APIs: Black-Box Data-Free Meta-Learningcited by 18
  15. 2025UCF-Crime-DVS: A Novel Event-Based Dataset for Video Anomaly Detection with Spiking Neural Networkscited by 18
  16. 2024Task Groupings Regularization: Data-Free Meta-Learning with Heterogeneous Pre-trained Modelscited by 12
  17. 2023LaSNN: Layer-wise ANN-to-SNN Distillation for Effective and Efficient Training in Deep Spiking Neural Networkscited by 11
  18. 2024BKDSNN: Enhancing the Performance of Learning-based Spiking Neural Networks Training with Blurred Knowledge Distillationcited by 11
  19. 2023Architecture, Dataset and Model-Scale Agnostic Data-free Meta-Learningcited by 10
  20. 2025Temporal Separation with Entropy Regularization for Knowledge Distillation in Spiking Neural Networkscited by 10
  21. 2024FREE: Faster and Better Data-Free Meta-Learningcited by 8
  22. 2025Efficient ANN-Guided Distillation: Aligning Rate-based Features of Spiking Neural Networks through Hybrid Block-wise Replacementcited by 6
  23. 2023Task-Distributionally Robust Data-Free Meta-Learningcited by 4
  24. 2025Cross Knowledge Distillation between Artificial and Spiking Neural Networkscited by 4

Built on by

Not cited yet.