# Pre-Defined Sparse Neural Networks with Hardware Acceleration

arXiv: https://arxiv.org/abs/1812.01164 · HTML: https://mlsanity.com/paper/1812.01164/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Dec 2018.

## Built on by (most cited first, 9 of 9)

- [Compute-Efficient Deep Learning: Algorithmic Trends and Opportunities](https://mlsanity.com/paper/2210.06640/) (2022)
- [Single-Shot Pruning for Offline Reinforcement Learning](https://mlsanity.com/paper/2112.15579/) (2021)
- [Sparse Training Theory for Scalable and Efficient Agents](https://mlsanity.com/paper/2103.01636/) (2021)
- [Sparse Bayesian Networks: Efficient Uncertainty Quantification in Medical Image Analysis](https://mlsanity.com/paper/2406.06946/) (2024)
- [Automated Pest Detection with DNN on the Edge for Precision Agriculture](https://mlsanity.com/paper/2108.00421/) (2021)
- [DRIVE: Dual Gradient-Based Rapid Iterative Pruning](https://mlsanity.com/paper/2404.03687/) (2024)
- [Artificial Neural Networks generated by Low Discrepancy Sequences](https://mlsanity.com/paper/2103.03543/) (2021)
- [Enhancing Split Computing and Early Exit Applications through Predefined Sparsity](https://mlsanity.com/paper/2407.11763/) (2024)
- [Asymptotic properties of one-layer artificial neural networks with sparse connectivity](https://mlsanity.com/paper/2112.00732/) (2021)

## Abstract (arXiv)

Neural networks have proven to be extremely powerful tools for modern artificial intelligence applications, but computational and storage complexity remain limiting factors. This paper presents two compatible contributions towards reducing the time, energy, computational, and storage complexities associated with multilayer perceptrons. Pre-defined sparsity is proposed to reduce the complexity during both training and inference, regardless of the implementation platform. Our results show that storage and computational complexity can be reduced by factors greater than 5X without significant performance loss. The second contribution is an architecture for hardware acceleration that is compatible with pre-defined sparsity. This architecture supports both training and inference modes and is flexible in the sense that it is not tied to a specific number of neurons. For example, this flexibility implies that various sized neural networks can be supported on various sized Field Programmable Gate Array (FPGA)s.