# Cross Knowledge Distillation between Artificial and Spiking Neural Networks

arXiv: https://arxiv.org/abs/2507.09269 · HTML: https://mlsanity.com/paper/2507.09269/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Distillation for spiking neural networks, one of 12 directions in Knowledge distillation.
- Builds on 19 of its 35 references that are in the map.
- First posted on arXiv 12 Jul 2025.

## Placed in

- Knowledge distillation / [Distillation for spiking neural networks](https://mlsanity.com/distill/distillation-for-spiking-neural-networks/)

## Builds on (most cited first, 19 of 19)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Similarity of Neural Network Representations Revisited](https://mlsanity.com/paper/1905.00414/) (2019)
- [Converting Static Image Datasets to Spiking Neuromorphic Datasets Using Saccades](https://mlsanity.com/paper/1507.07629/) (2015)
- [Unsupervised Event-based Learning of Optical Flow, Depth, and Egomotion](https://mlsanity.com/paper/1812.08156/) (2018)
- [HATS: Histograms of Averaged Time Surfaces for Robust Event-based Object Classification](https://mlsanity.com/paper/1803.07913/) (2018)
- [Knowledge Distillation from A Stronger Teacher](https://mlsanity.com/paper/2205.10536/) (2022)
- [Temporal Efficient Training of Spiking Neural Network via Gradient Re-weighting](https://mlsanity.com/paper/2202.11946/) (2022)
- [Neuromorphic Data Augmentation for Training Spiking Neural Networks](https://mlsanity.com/paper/2203.06145/) (2022)
- [SpinalNet: Deep Neural Network with Gradual Input](https://mlsanity.com/paper/2007.03347/) (2020)
- [Ternary Spike: Learning Ternary Spikes for Spiking Neural Networks](https://mlsanity.com/paper/2312.06372/) (2023)
- [BrainCog: A Spiking Neural Network based Brain-inspired Cognitive Intelligence Engine for Brain-inspired AI and Brain Simulation](https://mlsanity.com/paper/2207.08533/) (2022)
- [Distilling Spikes: Knowledge Distillation in Spiking Neural Networks](https://mlsanity.com/paper/2005.00288/) (2020)
- [UCF-Crime-DVS: A Novel Event-Based Dataset for Video Anomaly Detection with Spiking Neural Networks](https://mlsanity.com/paper/2503.12905/) (2025)
- [An Unsupervised STDP-based Spiking Neural Network Inspired By Biologically Plausible Learning Rules and Connections](https://mlsanity.com/paper/2207.02727/) (2022)
- [LaSNN: Layer-wise ANN-to-SNN Distillation for Effective and Efficient Training in Deep Spiking Neural Networks](https://mlsanity.com/paper/2304.09101/) (2023)
- [Improving Stability and Performance of Spiking Neural Networks through Enhancing Temporal Consistency](https://mlsanity.com/paper/2305.14174/) (2023)
- [An Efficient Knowledge Transfer Strategy for Spiking Neural Networks from Static to Event Domain](https://mlsanity.com/paper/2303.13077/) (2023)
- [Reduction of Class Activation Uncertainty with Background Information](https://mlsanity.com/paper/2305.03238/) (2023)
- [ReSpike: Residual Frames-based Hybrid Spiking Neural Networks for Efficient Action Recognition](https://mlsanity.com/paper/2409.01564/) (2024)

## Built on by (most cited first, 4 of 4)

- [Breaking the Modality Wall: Time-step Mixup for Efficient Spiking Knowledge Transfer from Static to Event Domain](https://mlsanity.com/paper/2511.12150/) (2025)
- [Sparse by Rule: Probability-Based N:M Pruning for Spiking Neural Networks](https://mlsanity.com/paper/2511.12097/) (2025)
- [STARS: Spike Tail-Aware Relational Synthesis for ANN-to-SNN Data-Free Knowledge Distillation](https://mlsanity.com/paper/2605.27409/) (2026)
- [ENCORE: Event-Assisted Complementary Motion Refinement for Learned Video Compression](https://mlsanity.com/paper/2607.28020/) (2026)

## Abstract (arXiv)

Recently, Spiking Neural Networks (SNNs) have demonstrated rich potential in computer vision domain due to their high biological plausibility, event-driven characteristic and energy-saving efficiency. Still, limited annotated event-based datasets and immature SNN architectures result in their performance inferior to that of Artificial Neural Networks (ANNs). To enhance the performance of SNNs on their optimal data format, DVS data, we explore using RGB data and well-performing ANNs to implement knowledge distillation. In this case, solving cross-modality and cross-architecture challenges is necessary. In this paper, we propose cross knowledge distillation (CKD), which not only leverages semantic similarity and sliding replacement to mitigate the cross-modality challenge, but also uses an indirect phased knowledge distillation to mitigate the cross-architecture challenge. We validated our method on main-stream neuromorphic datasets, including N-Caltech101 and CEP-DVS. The experimental results show that our method outperforms current State-of-the-Art methods. The code will be available at https://github.com/ShawnYE618/CKD