STARS: Spike Tail-Aware Relational Synthesis for ANN-to-SNN Data-Free Knowledge Distillation
Not on the map. Why?
Abstract (arXiv)
SNNs promise energy-efficient and low-latency inference, but their performance still trails that of ANNs. ANN-to-SNN knowledge distillation helps narrow this gap, yet the original training data are often unavailable in practical deployment settings. Existing data-free knowledge distillation (DFKD) methods synthesize surrogate data by matching teacher-side priors, especially BN statistics, but these ANN-oriented constraints mainly regularize mean and variance and therefore remain under-constrained for SNN students whose responses depend on threshold-crossing dynamics. In this paper, we propose Spike Tail-Aware Relational Synthesis (STARS), a plug-and-play method for ANN-to-SNN DFKD that augments standard BN-guided synthesis with two complementary objectives: Relational Consistency Alignment, which preserves cross-sample relational consistency between teacher and student, and Tail-Aware Regularization, which regularizes threshold-relevant tail probabilities through soft exceedance over teacher-derived thresholds. Together, these objectives generate synthetic batches that remain teacher-valid while becoming more informative for SNN students. Experiments on CIFAR-10, CIFAR-100, and Tiny-ImageNet across multiple ANN-SNN pairs show that our method consistently improves conventional DFKD baselines and even surpasses several KD methods, with gains of up to 4.6% on CIFAR-10 and 6.7% on CIFAR-100, highlighting the importance of complementing BN matching with relational and tail-aware constraints in SNN-oriented DFKD.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 36 of its 49 references that are in the map. First posted on arXiv 12 May 2026.
Builds on
36 of 49 references.
References · 36
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
- 2015Distilling the Knowledge in a Neural Networkcited by 10,076
- 2014FitNets: Hints for Thin Deep Netscited by 1,619
- 2016Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfercited by 1,059
- 2019Relational Knowledge Distillationcited by 873
- 2019Contrastive Representation Distillationcited by 734
- 2019Surrogate Gradient Learning in Spiking Neural Networkscited by 604
- 2019Dreaming to Distill: Data-free Knowledge Transfer via DeepInversioncited by 419
- 2017Spatio-Temporal Backpropagation for Training High-performance Spiking Neural Networkscited by 399
- 2020Incorporating Learnable Membrane Time Constant to Enhance Learning of Spiking Neural Networkscited by 340
- 2020Going Deeper With Directly-Trained Larger Spiking Neural Networkscited by 268
24 more
- 2022Spikformer: When Spiking Neural Network Meets Transformercited by 263
- 2019Data-Free Learning of Student Networkscited by 203
- 2021Optimal Conversion of Conventional Artificial Neural Networks to Spiking Neural Networkscited by 134
- 2019Zero-shot Knowledge Transfer via Adversarial Belief Matchingcited by 129
- 2019Zero-Shot Knowledge Distillation in Deep Networkscited by 119
- 2019Data-Free Adversarial Distillationcited by 90
- 2021Contrastive Model Inversion for Data-Free Knowledge Distillationcited by 66
- 2022GLIF: A Unified Gated Leaky Integrate-and-Fire Neuron for Spiking Neural Networkscited by 63
- 2020Progressive Tandem Learning for Pattern Recognition with Deep Spiking Neural Networkscited by 46
- 2022Robust and Resource-Efficient Data-Free Knowledge Distillation by Generative Pseudo Replaycited by 35
- 2023Joint A-SNN: Joint Training of Artificial and Spiking Neural Networks via Self-Distillation and Weight Factorizationcited by 34
- 2022Training Spiking Neural Networks with Local Tandem Learningcited by 22
- 2023NAYER: Noisy Layer Data Generation for Efficient and Effective Data-free Knowledge Distillationcited by 22
- 2023Learning to Learn from APIs: Black-Box Data-Free Meta-Learningcited by 18
- 2025UCF-Crime-DVS: A Novel Event-Based Dataset for Video Anomaly Detection with Spiking Neural Networkscited by 18
- 2024Task Groupings Regularization: Data-Free Meta-Learning with Heterogeneous Pre-trained Modelscited by 12
- 2023LaSNN: Layer-wise ANN-to-SNN Distillation for Effective and Efficient Training in Deep Spiking Neural Networkscited by 11
- 2024BKDSNN: Enhancing the Performance of Learning-based Spiking Neural Networks Training with Blurred Knowledge Distillationcited by 11
- 2023Architecture, Dataset and Model-Scale Agnostic Data-free Meta-Learningcited by 10
- 2025Temporal Separation with Entropy Regularization for Knowledge Distillation in Spiking Neural Networkscited by 10
- 2024FREE: Faster and Better Data-Free Meta-Learningcited by 8
- 2025Efficient ANN-Guided Distillation: Aligning Rate-based Features of Spiking Neural Networks through Hybrid Block-wise Replacementcited by 6
- 2023Task-Distributionally Robust Data-Free Meta-Learningcited by 4
- 2025Cross Knowledge Distillation between Artificial and Spiking Neural Networkscited by 4
Built on by
Not cited yet.