# Towards Accurate Binary Spiking Neural Networks: Learning with Adaptive Gradient Modulation Mechanism

arXiv: https://arxiv.org/abs/2502.14344 · HTML: https://mlsanity.com/paper/2502.14344/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 46 references that are in the map.
- First posted on arXiv 20 Feb 2025.

## Builds on (most cited first, 23 of 23)

- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation](https://mlsanity.com/paper/1308.3432/) (2013)
- [Identity Mappings in Deep Residual Networks](https://mlsanity.com/paper/1603.05027/) (2016)
- [Visualizing the Loss Landscape of Neural Nets](https://mlsanity.com/paper/1712.09913/) (2017)
- [XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks](https://mlsanity.com/paper/1603.05279/) (2016)
- [Sora: A Review on Background, Technology, Limitations, and Opportunities of Large Vision Models](https://mlsanity.com/paper/2402.17177/) (2024)
- [Spatio-Temporal Backpropagation for Training High-performance Spiking Neural Networks](https://mlsanity.com/paper/1706.02609/) (2017)
- [Bi-Real Net: Enhancing the Performance of 1-bit CNNs With Improved Representational Capability and Advanced Training Algorithm](https://mlsanity.com/paper/1808.00278/) (2018)
- [Direct Training for Spiking Neural Networks: Faster, Larger, Better](https://mlsanity.com/paper/1809.05793/) (2018)
- [Temporal Efficient Training of Spiking Neural Network via Gradient Re-weighting](https://mlsanity.com/paper/2202.11946/) (2022)
- [Additive Powers-of-Two Quantization: An Efficient Non-uniform Discretization for Neural Networks](https://mlsanity.com/paper/1909.13144/) (2019)
- [Three Mechanisms of Weight Decay Regularization](https://mlsanity.com/paper/1810.12281/) (2018)
- [Latent Weights Do Not Exist: Rethinking Binarized Neural Network Optimization](https://mlsanity.com/paper/1906.02107/) (2019)
- [ReCU: Reviving the Dead Weights in Binary Neural Networks](https://mlsanity.com/paper/2103.12369/) (2021)
- [Progressive Tandem Learning for Pattern Recognition with Deep Spiking Neural Networks](https://mlsanity.com/paper/2007.01204/) (2020)
- [Comprehensive SNN Compression Using ADMM Optimization and Activity Regularization](https://mlsanity.com/paper/1911.00822/) (2019)
- [Accurate LoRA-Finetuning Quantization of LLMs via Information Retention](https://mlsanity.com/paper/2402.05445/) (2024)
- [Exploring the Connection Between Binary and Spiking Neural Networks](https://mlsanity.com/paper/2002.10064/) (2020)
- [BS4NN: Binarized Spiking Neural Networks with Temporal Coding and Learning](https://mlsanity.com/paper/2007.04039/) (2020)
- [Towards Efficient Deep Spiking Neural Networks Construction with Spiking Activity based Pruning](https://mlsanity.com/paper/2406.01072/) (2024)
- [BiSNN: Training Spiking Neural Networks with Binary Weights via Bayesian Learning](https://mlsanity.com/paper/2012.08300/) (2020)
- [LitE-SNN: Designing Lightweight and Efficient Spiking Neural Network through Spatial-Temporal Compressive Network Search and Joint Optimization](https://mlsanity.com/paper/2401.14652/) (2024)
- [Neuromorphic Auditory Perception by Neural Spiketrum](https://mlsanity.com/paper/2309.05430/) (2023)

## Built on by (most cited first, 5 of 5)

- [AR-LIF: Adaptive reset leaky integrate-and-fire neuron for spiking neural networks](https://mlsanity.com/paper/2507.20746/) (2025)
- [Training-Free ANN-to-SNN Conversion for High-Performance Spiking Transformer](https://mlsanity.com/paper/2508.07710/) (2025)
- [BSO: Binary Spiking Online Optimization Algorithm](https://mlsanity.com/paper/2511.12502/) (2025)
- [A Self-Ensemble Inspired Approach for Effective Training of Binary-Weight Spiking Neural Networks](https://mlsanity.com/paper/2508.12609/) (2025)
- [Neuronal Self-Adaptation Enhances Capacity and Robustness of Representation in Spiking Neural Networks](https://mlsanity.com/paper/2603.20687/) (2026)

## Abstract (arXiv)

Binary Spiking Neural Networks (BSNNs) inherit the eventdriven paradigm of SNNs, while also adopting the reduced storage burden of binarization techniques. These distinct advantages grant BSNNs lightweight and energy-efficient characteristics, rendering them ideal for deployment on resource-constrained edge devices. However, due to the binary synaptic weights and non-differentiable spike function, effectively training BSNNs remains an open question. In this paper, we conduct an in-depth analysis of the challenge for BSNN learning, namely the frequent weight sign flipping problem. To mitigate this issue, we propose an Adaptive Gradient Modulation Mechanism (AGMM), which is designed to reduce the frequency of weight sign flipping by adaptively adjusting the gradients during the learning process. The proposed AGMM can enable BSNNs to achieve faster convergence speed and higher accuracy, effectively narrowing the gap between BSNNs and their full-precision equivalents. We validate AGMM on both static and neuromorphic datasets, and results indicate that it achieves state-of-the-art results among BSNNs. This work substantially reduces storage demands and enhances SNNs' inherent energy efficiency, making them highly feasible for resource-constrained environments.