# Fluctuation-driven initialization for spiking neural network training

arXiv: https://arxiv.org/abs/2206.10226 · HTML: https://mlsanity.com/paper/2206.10226/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Jun 2022.

## Built on by (most cited first, 16 of 16)

- [A Comprehensive Review of Spiking Neural Networks: Interpretation, Optimization, Efficiency, and Best Practices](https://mlsanity.com/paper/2303.10780/) (2023)
- [Stable Learning Using Spiking Neural Networks Equipped With Affine Encoders and Decoders](https://mlsanity.com/paper/2404.04549/) (2024)
- [Spiking Network Initialisation and Firing Rate Collapse](https://mlsanity.com/paper/2305.08879/) (2023)
- [Causal pieces: analysing and improving spiking neural networks piece by piece](https://mlsanity.com/paper/2504.14015/) (2025)
- [Spiking mode-based neural networks](https://mlsanity.com/paper/2310.14621/) (2023)
- [Exploring neural oscillations during speech perception via surrogate gradient spiking neural networks](https://mlsanity.com/paper/2404.14024/) (2024)
- [Decoding finger velocity from cortical spike trains with recurrent spiking neural networks](https://mlsanity.com/paper/2409.01762/) (2024)
- [Fundamental Survey on Neuromorphic Based Audio Classification](https://mlsanity.com/paper/2502.15056/) (2025)
- [Hardware-Accelerated Event-Graph Neural Networks for Low-Latency Time-Series Classification on SoC FPGA](https://mlsanity.com/paper/2503.06629/) (2025)
- [FLAMES: A Hybrid Spiking-State Space Model for Adaptive Memory Retention in Event-Based Learning](https://mlsanity.com/paper/2504.01257/) (2025)
- [Scaling Laws and Tradeoffs in Recurrent Networks of Expressive Neurons](https://mlsanity.com/paper/2605.12049/) (2026)
- [Deep activity propagation via weight initialization in spiking neural networks](https://mlsanity.com/paper/2410.00580/) (2024)
- [Learning with Spike Synchrony in Spiking Neural Networks](https://mlsanity.com/paper/2505.14841/) (2025)
- [Spiking Neural Networks for Continuous Control via End-to-End Model-Based Learning](https://mlsanity.com/paper/2509.05356/) (2025)
- [Random Feature Spiking Neural Networks](https://mlsanity.com/paper/2510.01012/) (2025)
- [End-to-End Keyword Spotting on FPGA Using Graph Neural Networks with a Neuromorphic Auditory Sensor](https://mlsanity.com/paper/2605.09570/) (2026)

## Abstract (arXiv)

Spiking neural networks (SNNs) underlie low-power, fault-tolerant information processing in the brain and could constitute a power-efficient alternative to conventional deep neural networks when implemented on suitable neuromorphic hardware accelerators. However, instantiating SNNs that solve complex computational tasks in-silico remains a significant challenge. Surrogate gradient (SG) techniques have emerged as a standard solution for training SNNs end-to-end. Still, their success depends on synaptic weight initialization, similar to conventional artificial neural networks (ANNs). Yet, unlike in the case of ANNs, it remains elusive what constitutes a good initial state for an SNN. Here, we develop a general initialization strategy for SNNs inspired by the fluctuation-driven regime commonly observed in the brain. Specifically, we derive practical solutions for data-dependent weight initialization that ensure fluctuation-driven firing in the widely used leaky integrate-and-fire (LIF) neurons. We empirically show that SNNs initialized following our strategy exhibit superior learning performance when trained with SGs. These findings generalize across several datasets and SNN architectures, including fully connected, deep convolutional, recurrent, and more biologically plausible SNNs obeying Dale's law. Thus fluctuation-driven initialization provides a practical, versatile, and easy-to-implement strategy for improving SNN training performance on diverse tasks in neuromorphic engineering and computational neuroscience.