# Sparse-firing regularization methods for spiking neural networks with time-to-first spike coding

arXiv: https://arxiv.org/abs/2307.13007 · HTML: https://mlsanity.com/paper/2307.13007/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 58 references that are in the map.
- First posted on arXiv 24 Jul 2023.

## Builds on (most cited first, 28 of 28)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Surrogate Gradient Learning in Spiking Neural Networks](https://mlsanity.com/paper/1901.09948/) (2019)
- [Spatio-Temporal Backpropagation for Training High-performance Spiking Neural Networks](https://mlsanity.com/paper/1706.02609/) (2017)
- [Deep Learning in Spiking Neural Networks](https://mlsanity.com/paper/1804.08150/) (2018)
- [Training Spiking Neural Networks Using Lessons From Deep Learning](https://mlsanity.com/paper/2109.12894/) (2021)
- [SuperSpike: Supervised learning in multi-layer spiking neural networks](https://mlsanity.com/paper/1705.11146/) (2017)
- [Spiking-YOLO: Spiking Neural Network for Energy-Efficient Object Detection](https://mlsanity.com/paper/1903.06530/) (2019)
- [Accurate and efficient time-domain classification with adaptive spiking recurrent neural networks](https://mlsanity.com/paper/2103.12593/) (2021)
- [Machine Learning at the Network Edge: A Survey](https://mlsanity.com/paper/1908.00080/) (2019)
- [Revisiting Batch Normalization for Training Low-latency Deep Spiking Neural Networks from Scratch](https://mlsanity.com/paper/2010.01729/) (2020)
- [Supervised learning based on temporal coding in spiking neural networks](https://mlsanity.com/paper/1606.08165/) (2016)
- [Event-Based Backpropagation can compute Exact Gradients for Spiking Neural Networks](https://mlsanity.com/paper/2009.08378/) (2020)
- [A scalable multi-core architecture with heterogeneous memory structures for Dynamic Neuromorphic Asynchronous Processors (DYNAPs)](https://mlsanity.com/paper/1708.04198/) (2017)
- [Fast and energy-efficient neuromorphic deep learning with first-spike times](https://mlsanity.com/paper/1912.11443/) (2019)
- [S4NN: temporal backpropagation for spiking neural networks with one spike per neuron](https://mlsanity.com/paper/1910.09495/) (2019)
- [Gradient Descent for Spiking Neural Networks](https://mlsanity.com/paper/1706.04698/) (2017)
- [Temporal-Coded Deep Spiking Neural Network with Easy Training and Robust Performance](https://mlsanity.com/paper/1909.10837/) (2019)
- [Neuromorphic Deep Learning Machines](https://mlsanity.com/paper/1612.05596/) (2016)
- [Surrogate gradients for analog neuromorphic computing](https://mlsanity.com/paper/2006.07239/) (2020)
- [BS4NN: Binarized Spiking Neural Networks with Temporal Coding and Learning](https://mlsanity.com/paper/2007.04039/) (2020)
- [StereoSpike: Depth Learning with a Spiking Neural Network](https://mlsanity.com/paper/2109.13751/) (2021)
- [Low-activity supervised convolutional spiking neural networks applied to speech commands recognition](https://mlsanity.com/paper/2011.06846/) (2020)
- [Adaptive Sparse Structure Development with Pruning and Regeneration for Spiking Neural Networks](https://mlsanity.com/paper/2211.12219/) (2022)
- [A Supervised Learning Algorithm for Multilayer Spiking Neural Networks Based on Temporal Coding Toward Energy-Efficient VLSI Processor Design](https://mlsanity.com/paper/2001.05348/) (2020)
- [Hardware Implementation of Spiking Neural Networks Using Time-To-First-Spike Encoding](https://mlsanity.com/paper/2006.05033/) (2020)
- [Timing-Based Backpropagation in Spiking Neural Networks Without Single-Spike Restrictions](https://mlsanity.com/paper/2211.16113/) (2022)
- [Effects of VLSI Circuit Constraints on Temporal-Coding Multilayer Spiking Neural Networks](https://mlsanity.com/paper/2106.10382/) (2021)

## Built on by (most cited first, 3 of 3)

- [Harnessing Nonidealities in Analog In-Memory Computing Circuits: A Physical Modeling Approach for Neuromorphic Systems](https://mlsanity.com/paper/2412.09010/) (2024)
- [Sparsity-Aware Hardware-Software Co-Design of Spiking Neural Networks: An Overview](https://mlsanity.com/paper/2408.14437/) (2024)
- [Matterhorn: Masked Time-to-First-Spike Encoding by Reassigning the Silent State for Sparse and Energy-Efficient Spiking Transformers](https://mlsanity.com/paper/2601.22876/) (2026)

## Abstract (arXiv)

The training of multilayer spiking neural networks (SNNs) using the error backpropagation algorithm has made significant progress in recent years. Among the various training schemes, the error backpropagation method that directly uses the firing time of neurons has attracted considerable attention because it can realize ideal temporal coding. This method uses time-to-first spike (TTFS) coding, in which each neuron fires at most once, and this restriction on the number of firings enables information to be processed at a very low firing frequency. This low firing frequency increases the energy efficiency of information processing in SNNs, which is important not only because of its similarity with information processing in the brain, but also from an engineering point of view. However, only an upper limit has been provided for TTFS-coded SNNs, and the information-processing capability of SNNs at lower firing frequencies has not been fully investigated. In this paper, we propose two spike timing-based sparse-firing (SSR) regularization methods to further reduce the firing frequency of TTFS-coded SNNs. The first is the membrane potential-aware SSR (M-SSR) method, which has been derived as an extreme form of the loss function of the membrane potential value. The second is the firing condition-aware SSR (F-SSR) method, which is a regularization function obtained from the firing conditions. Both methods are characterized by the fact that they only require information about the firing timing and associated weights. The effects of these regularization methods were investigated on the MNIST, Fashion-MNIST, and CIFAR-10 datasets using multilayer perceptron networks and convolutional neural network structures.