# To prune or not to prune : A chaos-causality approach to principled pruning of dense neural networks

arXiv: https://arxiv.org/abs/2308.09955 · HTML: https://mlsanity.com/paper/2308.09955/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 26 references that are in the map.
- First posted on arXiv 19 Aug 2023.

## Builds on (most cited first, 11 of 11)

- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks](https://mlsanity.com/paper/1803.03635/) (2018)
- [Rethinking the Value of Network Pruning](https://mlsanity.com/paper/1810.05270/) (2018)
- [Stochastic Gradient Descent Optimizes Over-parameterized Deep ReLU Networks](https://mlsanity.com/paper/1811.08888/) (2018)
- [Predicting trends in the quality of state-of-the-art neural networks without access to training or testing data](https://mlsanity.com/paper/2002.06716/) (2020)
- [The Unreasonable Effectiveness of Random Pruning: Return of the Most Naive Baseline for Sparse Training](https://mlsanity.com/paper/2202.02643/) (2022)
- [Nonlinear Approximation via Compositions](https://mlsanity.com/paper/1902.10170/) (2019)
- [A Probabilistic Approach to Neural Network Pruning](https://mlsanity.com/paper/2105.10065/) (2021)
- [ChaosNet: A Chaos based Artificial Neural Network Architecture for Classification](https://mlsanity.com/paper/1910.02423/) (2019)
- [Edge of chaos as a guiding principle for modern neural network training](https://mlsanity.com/paper/2107.09437/) (2021)
- [AdaSwarm: Augmenting Gradient-Based optimizers in Deep Learning with Swarm Intelligence](https://mlsanity.com/paper/2006.09875/) (2020)

## Abstract (arXiv)

Reducing the size of a neural network (pruning) by removing weights without impacting its performance is an important problem for resource-constrained devices. In the past, pruning was typically accomplished by ranking or penalizing weights based on criteria like magnitude and removing low-ranked weights before retraining the remaining ones. Pruning strategies may also involve removing neurons from the network in order to achieve the desired reduction in network size. We formulate pruning as an optimization problem with the objective of minimizing misclassifications by selecting specific weights. To accomplish this, we have introduced the concept of chaos in learning (Lyapunov exponents) via weight updates and exploiting causality to identify the causal weights responsible for misclassification. Such a pruned network maintains the original performance and retains feature explainability.