# Sparsifying Spiking Networks through Local Rhythms

arXiv: https://arxiv.org/abs/2305.10191 · HTML: https://mlsanity.com/paper/2305.10191/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 8 of its 19 references that are in the map.
- First posted on arXiv 30 Apr 2023.

## Builds on (most cited first, 8 of 8)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [The State of Sparsity in Deep Neural Networks](https://mlsanity.com/paper/1902.09574/) (2019)
- [The Lazy Neuron Phenomenon: On Emergence of Activation Sparsity in Transformers](https://mlsanity.com/paper/2210.06313/) (2022)
- [SPDF: Sparse Pre-training and Dense Fine-tuning for Large Language Models](https://mlsanity.com/paper/2303.10464/) (2023)
- [Deep Phasor Networks: Connecting Conventional and Spiking Neural Networks](https://mlsanity.com/paper/2106.11908/) (2021)
- [Deep Learning in Spiking Phasor Neural Networks](https://mlsanity.com/paper/2204.00507/) (2022)
- [Residual and Attentional Architectures for Vector-Symbols](https://mlsanity.com/paper/2207.08953/) (2022)

## Abstract (arXiv)

It has been well-established that within conventional neural networks, many of the values produced at each layer are zero. In this work, I demonstrate that spiking neural networks can prevent the transmission of spikes representing values close to zero using local information. This can reduce the amount of energy required for communication and computation in these networks while preserving accuracy. Additionally, this demonstrates a novel application of biologically observed spiking rhythms.