# Optimizing the Consumption of Spiking Neural Networks with Activity Regularization

arXiv: https://arxiv.org/abs/2204.01460 · HTML: https://mlsanity.com/paper/2204.01460/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 21 references that are in the map.
- First posted on arXiv 4 Apr 2022.

## Builds on (most cited first, 13 of 13)

- [Knowledge Distillation: A Survey](https://mlsanity.com/paper/2006.05525/) (2020)
- [Learning Efficient Convolutional Networks through Network Slimming](https://mlsanity.com/paper/1708.06519/) (2017)
- [Surrogate Gradient Learning in Spiking Neural Networks](https://mlsanity.com/paper/1901.09948/) (2019)
- [Learning Sparse Neural Networks through L0 Regularization](https://mlsanity.com/paper/1712.01312/) (2017)
- [Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networks](https://mlsanity.com/paper/2102.00554/) (2021)
- [EIE: Efficient Inference Engine on Compressed Deep Neural Network](https://mlsanity.com/paper/1602.01528/) (2016)
- [Pruning and Quantization for Deep Neural Network Acceleration: A Survey](https://mlsanity.com/paper/2101.09671/) (2021)
- [SCNN: An Accelerator for Compressed-sparse Convolutional Neural Networks](https://mlsanity.com/paper/1708.04485/) (2017)
- [DeepHoyer: Learning Sparser Neural Network with Differentiable Scale-Invariant Sparsity Measures](https://mlsanity.com/paper/1908.09979/) (2019)
- [Optimizing the energy consumption of spiking neural networks for neuromorphic applications](https://mlsanity.com/paper/1912.01268/) (2019)
- [Accelerating Convolutional Neural Networks via Activation Map Compression](https://mlsanity.com/paper/1812.04056/) (2018)
- [Low-activity supervised convolutional spiking neural networks applied to speech commands recognition](https://mlsanity.com/paper/2011.06846/) (2020)
- [Training for temporal sparsity in deep neural networks, application in video processing](https://mlsanity.com/paper/2107.07305/) (2021)

## Built on by (most cited first, 4 of 4)

- [Spiking Neural Network for Ultra-low-latency and High-accurate Object Detection](https://mlsanity.com/paper/2306.12010/) (2023)
- [Spiking Synaptic Penalty: Appropriate Penalty Term for Energy-Efficient Spiking Neural Networks](https://mlsanity.com/paper/2302.01500/) (2023)
- [On Reducing Activity with Distillation and Regularization for Energy Efficient Spiking Neural Networks](https://mlsanity.com/paper/2406.18350/) (2024)
- [Enhancing Eye Feature Estimation from Event Data Streams through Adaptive Inference State Space Modeling](https://mlsanity.com/paper/2603.14077/) (2026)

## Abstract (arXiv)

Reducing energy consumption is a critical point for neural network models running on edge devices. In this regard, reducing the number of multiply-accumulate (MAC) operations of Deep Neural Networks (DNNs) running on edge hardware accelerators will reduce the energy consumption during inference. Spiking Neural Networks (SNNs) are an example of bio-inspired techniques that can further save energy by using binary activations, and avoid consuming energy when not spiking. The networks can be configured for equivalent accuracy on a task through DNN-to-SNN conversion frameworks but their conversion is based on rate coding therefore the synaptic operations can be high. In this work, we look into different techniques to enforce sparsity on the neural network activation maps and compare the effect of different training regularizers on the efficiency of the optimized DNNs and SNNs.