# Real Spike: Learning Real-valued Spikes for Spiking Neural Networks

arXiv: https://arxiv.org/abs/2210.06686 · HTML: https://mlsanity.com/paper/2210.06686/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 43 references that are in the map.
- First posted on arXiv 13 Oct 2022.

## Builds on (most cited first, 21 of 21)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [Surrogate Gradient Learning in Spiking Neural Networks](https://mlsanity.com/paper/1901.09948/) (2019)
- [Incorporating Learnable Membrane Time Constant to Enhance Learning of Spiking Neural Networks](https://mlsanity.com/paper/2007.05785/) (2020)
- [Deep Residual Learning in Spiking Neural Networks](https://mlsanity.com/paper/2102.04159/) (2021)
- [Going Deeper With Directly-Trained Larger Spiking Neural Networks](https://mlsanity.com/paper/2011.05280/) (2020)
- [SLAYER: Spike Layer Error Reassignment in Time](https://mlsanity.com/paper/1810.08646/) (2018)
- [Going Deeper in Spiking Neural Networks: VGG and Residual Architectures](https://mlsanity.com/paper/1802.02627/) (2018)
- [Direct Training for Spiking Neural Networks: Faster, Larger, Better](https://mlsanity.com/paper/1809.05793/) (2018)
- [Enabling Deep Spiking Neural Networks with Hybrid Conversion and Spike Timing Dependent Backpropagation](https://mlsanity.com/paper/2005.01807/) (2020)
- [Enabling Spike-based Backpropagation for Training Deep Neural Network Architectures](https://mlsanity.com/paper/1903.06379/) (2019)
- [A Free Lunch From ANN: Towards Efficient, Accurate Spiking Neural Networks Calibration](https://mlsanity.com/paper/2106.06984/) (2021)
- [Temporal Spike Sequence Learning via Backpropagation for Deep Spiking Neural Networks](https://mlsanity.com/paper/2002.10085/) (2020)
- [DIET-SNN: Direct Input Encoding With Leakage and Threshold Optimization in Deep Spiking Neural Networks](https://mlsanity.com/paper/2008.03658/) (2020)
- [Gradient Descent for Spiking Neural Networks](https://mlsanity.com/paper/1706.04698/) (2017)
- [T2FSNN: Deep Spiking Neural Networks with Time-to-first-spike Coding](https://mlsanity.com/paper/2003.11741/) (2020)
- [Exploring the Connection Between Binary and Spiking Neural Networks](https://mlsanity.com/paper/2002.10064/) (2020)
- [A Biologically Plausible Supervised Learning Method for Spiking Neural Networks Using the Symmetric STDP Rule](https://mlsanity.com/paper/1812.06574/) (2018)
- [Fast and Efficient Information Transmission with Burst Spikes in Deep Spiking Neural Networks](https://mlsanity.com/paper/1809.03142/) (2018)

## Built on by (most cited first, 22 of 22)

- [Joint A-SNN: Joint Training of Artificial and Spiking Neural Networks via Self-Distillation and Weight Factorization](https://mlsanity.com/paper/2305.02099/) (2023)
- [Membrane Potential Batch Normalization for Spiking Neural Networks](https://mlsanity.com/paper/2308.08359/) (2023)
- [Spiking PointNet: Spiking Neural Networks for Point Clouds](https://mlsanity.com/paper/2310.06232/) (2023)
- [Direct Learning-Based Deep Spiking Neural Networks: A Review](https://mlsanity.com/paper/2305.19725/) (2023)
- [RMP-Loss: Regularizing Membrane Potential Distribution for Spiking Neural Networks](https://mlsanity.com/paper/2308.06787/) (2023)
- [Take A Shortcut Back: Mitigating the Gradient Vanishing for Training Spiking Neural Networks](https://mlsanity.com/paper/2401.04486/) (2024)
- [Spiking Transformer:Introducing Accurate Addition-Only Spiking Self-Attention for Transformer](https://mlsanity.com/paper/2503.00226/) (2025)
- [Biologically inspired structure learning with reverse knowledge distillation for spiking neural networks](https://mlsanity.com/paper/2304.09500/) (2023)
- [STAA-SNN: Spatial-Temporal Attention Aggregator for Spiking Neural Networks](https://mlsanity.com/paper/2503.02689/) (2025)
- [Self-Distillation Learning Based on Temporal-Spatial Consistency for Spiking Neural Networks](https://mlsanity.com/paper/2406.07862/) (2024)
- [Autaptic Synaptic Circuit Enhances Spatio-temporal Predictive Learning of Spiking Neural Networks](https://mlsanity.com/paper/2406.00405/) (2024)
- [MT-SNN: Enhance Spiking Neural Network with Multiple Thresholds](https://mlsanity.com/paper/2303.11127/) (2023)
- [Efficient ANN-Guided Distillation: Aligning Rate-based Features of Spiking Neural Networks through Hybrid Block-wise Replacement](https://mlsanity.com/paper/2503.16572/) (2025)
- [Deep Multi-Threshold Spiking-UNet for Image Processing](https://mlsanity.com/paper/2307.10974/) (2023)
- [TS-SNN: Temporal Shift Module for Spiking Neural Networks](https://mlsanity.com/paper/2505.04165/) (2025)
- [ReverB-SNN: Reversing Bit of the Weight and Activation for Spiking Neural Networks](https://mlsanity.com/paper/2506.07720/) (2025)
- [Towards Efficient and Accurate Spiking Neural Networks via Adaptive Bit Allocation](https://mlsanity.com/paper/2506.23717/) (2025)
- [Brain-inspired Evolutionary Architectures for Spiking Neural Networks](https://mlsanity.com/paper/2309.05263/) (2023)
- [DTA: Dual Temporal-channel-wise Attention for Spiking Neural Networks](https://mlsanity.com/paper/2503.10052/) (2025)
- [Integer Binary-Range Alignment Neuron for Spiking Neural Networks](https://mlsanity.com/paper/2506.05679/) (2025)
- [Learnable Burst Quantization for Expressive and Efficient Spiking Neural Networks](https://mlsanity.com/paper/2604.25688/) (2026)
- [SAFformer:Improving Spiking Transformer via Active Predictive Filtering](https://mlsanity.com/paper/2605.08270/) (2026)

## Abstract (arXiv)

Brain-inspired spiking neural networks (SNNs) have recently drawn more and more attention due to their event-driven and energy-efficient characteristics. The integration of storage and computation paradigm on neuromorphic hardwares makes SNNs much different from Deep Neural Networks (DNNs). In this paper, we argue that SNNs may not benefit from the weight-sharing mechanism, which can effectively reduce parameters and improve inference efficiency in DNNs, in some hardwares, and assume that an SNN with unshared convolution kernels could perform better. Motivated by this assumption, a training-inference decoupling method for SNNs named as Real Spike is proposed, which not only enjoys both unshared convolution kernels and binary spikes in inference-time but also maintains both shared convolution kernels and Real-valued Spikes during training. This decoupling mechanism of SNN is realized by a re-parameterization technique. Furthermore, based on the training-inference-decoupled idea, a series of different forms for implementing Real Spike on different levels are presented, which also enjoy shared convolutions in the inference and are friendly to both neuromorphic and non-neuromorphic hardware platforms. A theoretical proof is given to clarify that the Real Spike-based SNN network is superior to its vanilla counterpart. Experimental results show that all different Real Spike versions can consistently improve the SNN performance. Moreover, the proposed method outperforms the state-of-the-art models on both non-spiking static and neuromorphic datasets.