# Spiking mode-based neural networks

arXiv: https://arxiv.org/abs/2310.14621 · HTML: https://mlsanity.com/paper/2310.14621/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 35 references that are in the map.
- First posted on arXiv 23 Oct 2023.

## Builds on (most cited first, 14 of 14)

- [Surrogate Gradient Learning in Spiking Neural Networks](https://mlsanity.com/paper/1901.09948/) (2019)
- [SuperSpike: Supervised learning in multi-layer spiking neural networks](https://mlsanity.com/paper/1705.11146/) (2017)
- [The Heidelberg spiking datasets for the systematic evaluation of spiking neural networks](https://mlsanity.com/paper/1910.07407/) (2019)
- [Linking connectivity, dynamics and computations in low-rank recurrent neural networks](https://mlsanity.com/paper/1711.09672/) (2017)
- [Interpreting neural computations by examining intrinsic and embedding dimensionality of neural activity](https://mlsanity.com/paper/2107.04084/) (2021)
- [Fluctuation-driven initialization for spiking neural network training](https://mlsanity.com/paper/2206.10226/) (2022)
- [Supervised Learning in Spiking Neural Networks with FORCE Training](https://mlsanity.com/paper/1609.02545/) (2016)
- [Emergence of hierarchical modes from deep learning](https://mlsanity.com/paper/2208.09859/) (2022)
- [Learning recurrent dynamics in spiking networks](https://mlsanity.com/paper/1803.06622/) (2018)
- [Spectrum of non-Hermitian deep-Hebbian neural networks](https://mlsanity.com/paper/2208.11411/) (2022)
- [Learning universal computations with spikes](https://mlsanity.com/paper/1505.07866/) (2015)
- [Using Firing-Rate Dynamics to Train Recurrent Networks of Spiking Model Neurons](https://mlsanity.com/paper/1601.07620/) (2016)
- [Associative memory model with arbitrary Hebbian length](https://mlsanity.com/paper/2103.14317/) (2021)
- [Approximating nonlinear functions with latent boundaries in low-rank excitatory-inhibitory spiking networks](https://mlsanity.com/paper/2307.09334/) (2023)

## Built on by (most cited first, 1 of 1)

- [Optimization of Low-Latency Spiking Neural Networks Utilizing Historical Dynamics of Refractory Periods](https://mlsanity.com/paper/2507.02960/) (2025)

## Abstract (arXiv)

Spiking neural networks play an important role in brain-like neuromorphic computations and in studying working mechanisms of neural circuits. One drawback of training a large scale spiking neural network is that updating all weights is quite expensive. Furthermore, after training, all information related to the computational task is hidden into the weight matrix, prohibiting us from a transparent understanding of circuit mechanisms. Therefore, in this work, we address these challenges by proposing a spiking mode-based training protocol, where the recurrent weight matrix is explained as a Hopfield-like multiplication of three matrices: input, output modes and a score matrix. The first advantage is that the weight is interpreted by input and output modes and their associated scores characterizing the importance of each decomposition term. The number of modes is thus adjustable, allowing more degrees of freedom for modeling the experimental data. This significantly reduces the training cost because of significantly reduced space complexity for learning. Training spiking networks is thus carried out in the mode-score space. The second advantage is that one can project the high dimensional neural activity (filtered spike train) in the state space onto the mode space which is typically of a low dimension, e.g., a few modes are sufficient to capture the shape of the underlying neural manifolds. We successfully apply our framework in two computational tasks -- digit classification and selective sensory integration tasks. Our method accelerate the training of spiking neural networks by a Hopfield-like decomposition, and moreover this training leads to low-dimensional attractor structures of high-dimensional neural dynamics.