# NoiseOut: A Simple Way to Prune Neural Networks

arXiv: https://arxiv.org/abs/1611.06211 · HTML: https://mlsanity.com/paper/1611.06211/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Nov 2016.

## Built on by (most cited first, 9 of 9)

- [A Simple and Effective Pruning Approach for Large Language Models](https://mlsanity.com/paper/2306.11695/) (2023)
- [DICE: Leveraging Sparsification for Out-of-Distribution Detection](https://mlsanity.com/paper/2111.09805/) (2021)
- [SingleSketch2Mesh : Generating 3D Mesh model from Sketch](https://mlsanity.com/paper/2203.03157/) (2022)
- [Detecting and Learning Out-of-Distribution Data in the Open world: Algorithm and Theory](https://mlsanity.com/paper/2310.06221/) (2023)
- [Linearity-based neural network compression](https://mlsanity.com/paper/2506.21146/) (2025)
- [Elimination-compensation pruning for fully-connected neural networks](https://mlsanity.com/paper/2602.20467/) (2026)
- [Solving hybrid machine learning tasks by traversing weight space geodesics](https://mlsanity.com/paper/2106.02793/) (2021)
- [Catalyst: Out-of-Distribution Detection via Elastic Scaling](https://mlsanity.com/paper/2602.02409/) (2026)
- [Plasticity of Growing and Elastic Neural Networks in Online Continual Learning](https://mlsanity.com/paper/2608.01475/) (2026)

## Abstract (arXiv)

Neural networks are usually over-parameterized with significant redundancy in the number of required neurons which results in unnecessary computation and memory usage at inference time. One common approach to address this issue is to prune these big networks by removing extra neurons and parameters while maintaining the accuracy. In this paper, we propose NoiseOut, a fully automated pruning algorithm based on the correlation between activations of neurons in the hidden layers. We prove that adding additional output neurons with entirely random targets results into a higher correlation between neurons which makes pruning by NoiseOut even more efficient. Finally, we test our method on various networks and datasets. These experiments exhibit high pruning rates while maintaining the accuracy of the original network.