# Advancing Direct Training for Spiking Neural Networks with Circulate-Firing Neurons and Learnable Gradients

arXiv: https://arxiv.org/abs/2605.27412 · HTML: https://mlsanity.com/paper/2605.27412/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 43 of its 71 references that are in the map.
- First posted on arXiv 14 May 2026.

## Builds on (most cited first, 30 of 43)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Convolutional Neural Networks for Sentence Classification](https://mlsanity.com/paper/1408.5882/) (2014)
- [An overview of gradient descent optimization algorithms](https://mlsanity.com/paper/1609.04747/) (2016)
- [Surrogate Gradient Learning in Spiking Neural Networks](https://mlsanity.com/paper/1901.09948/) (2019)
- [Spatio-Temporal Backpropagation for Training High-performance Spiking Neural Networks](https://mlsanity.com/paper/1706.02609/) (2017)
- [Incorporating Learnable Membrane Time Constant to Enhance Learning of Spiking Neural Networks](https://mlsanity.com/paper/2007.05785/) (2020)
- [Deep Residual Learning in Spiking Neural Networks](https://mlsanity.com/paper/2102.04159/) (2021)
- [Going Deeper With Directly-Trained Larger Spiking Neural Networks](https://mlsanity.com/paper/2011.05280/) (2020)
- [Spikformer: When Spiking Neural Network Meets Transformer](https://mlsanity.com/paper/2209.15425/) (2022)
- [SLAYER: Spike Layer Error Reassignment in Time](https://mlsanity.com/paper/1810.08646/) (2018)
- [Direct Training for Spiking Neural Networks: Faster, Larger, Better](https://mlsanity.com/paper/1809.05793/) (2018)
- [Training Deep Spiking Neural Networks using Backpropagation](https://mlsanity.com/paper/1608.08782/) (2016)
- [Temporal Efficient Training of Spiking Neural Network via Gradient Re-weighting](https://mlsanity.com/paper/2202.11946/) (2022)
- [Long short-term memory and learning-to-learn in networks of spiking neurons](https://mlsanity.com/paper/1803.09574/) (2018)
- [End-to-End Model-Free Reinforcement Learning for Urban Driving using Implicit Affordances](https://mlsanity.com/paper/1911.10868/) (2019)
- [RMP-SNN: Residual Membrane Potential Neuron for Enabling Deeper High-Accuracy and Low-Latency Spiking Neural Network](https://mlsanity.com/paper/2003.01811/) (2020)
- [Optimal ANN-SNN Conversion for High-accuracy and Ultra-low-latency Spiking Neural Networks](https://mlsanity.com/paper/2303.04347/) (2023)
- [A Free Lunch From ANN: Towards Efficient, Accurate Spiking Neural Networks Calibration](https://mlsanity.com/paper/2106.06984/) (2021)
- [Revisiting Batch Normalization for Training Low-latency Deep Spiking Neural Networks from Scratch](https://mlsanity.com/paper/2010.01729/) (2020)
- [DIET-SNN: Direct Input Encoding With Leakage and Threshold Optimization in Deep Spiking Neural Networks](https://mlsanity.com/paper/2008.03658/) (2020)
- [GLIF: A Unified Gated Leaky Integrate-and-Fire Neuron for Spiking Neural Networks](https://mlsanity.com/paper/2210.13768/) (2022)
- [Neuromorphic Data Augmentation for Training Spiking Neural Networks](https://mlsanity.com/paper/2203.06145/) (2022)
- [Parallel Spiking Neurons with High Efficiency and Ability to Learn Long-term Dependencies](https://mlsanity.com/paper/2304.12760/) (2023)
- [Bridging the Gap between ANNs and SNNs by Calibrating Offset Spikes](https://mlsanity.com/paper/2302.10685/) (2023)
- [Ternary Spike: Learning Ternary Spikes for Spiking Neural Networks](https://mlsanity.com/paper/2312.06372/) (2023)
- [TCJA-SNN: Temporal-Channel Joint Attention for Spiking Neural Networks](https://mlsanity.com/paper/2206.10177/) (2022)
- [Membrane Potential Batch Normalization for Spiking Neural Networks](https://mlsanity.com/paper/2308.08359/) (2023)
- [RMP-Loss: Regularizing Membrane Potential Distribution for Spiking Neural Networks](https://mlsanity.com/paper/2308.06787/) (2023)
- [Training Spiking Neural Networks with Local Tandem Learning](https://mlsanity.com/paper/2210.04532/) (2022)

## Built on by (most cited first, 1 of 1)

- [SAGE: Surrogate-gradient Adaptation via Attention-Guided Entropy for Spiking Transformers](https://mlsanity.com/paper/2608.13702/) (2026)

## Abstract (arXiv)

Spiking Neural Networks (SNNs) have emerged with promising energy-efficient property, yet a substantial performance gap persists compared to Artificial Neural Networks (ANNs). This gap stems from at least two key limitations: first, conventional spiking neurons offer limited information representation capacity, underutilizing the rich dynamics of membrane potentials; second, fixed surrogate gradient (SG) functions across time steps leads to imprecise gradient propagation, impeding effective direct training. To address these two challenges, we propose a new direct training algorithm with three core innovations: first, a circulate-firing spiking neuron model that enhances information representation capacity by leveraging membrane potentials more effectively; second, a time-step-wise learnable surrogate gradient function, enabling accurate gradient estimation during backpropagation; third, a positive-negative balanced loss function to achieve equilibrium between positive and negative membrane potentials and further boost SNN performance. Extensive experiments demonstrate that our methods achieve competitive performance across multiple datasets. Our methods can generalize seamlessly to advanced architectures of Transformer, consistently outperforming existing methods. Our work highlights the effectiveness of further harnessing intrinsic membrane dynamics of SNNs for performance improvement, and thus open a new avenue for advancing high-performance spiking neural architectures.