# SiPPing Neural Networks: Sensitivity-informed Provable Pruning of Neural Networks

arXiv: https://arxiv.org/abs/1910.05422 · HTML: https://mlsanity.com/paper/1910.05422/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Oct 2019.

## Built on by (most cited first, 17 of 17)

- [Lost in Pruning: The Effects of Pruning Neural Networks beyond Test Accuracy](https://mlsanity.com/paper/2103.03014/) (2021)
- [A Winning Hand: Compressing Deep Networks Can Improve Out-Of-Distribution Robustness](https://mlsanity.com/paper/2106.09129/) (2021)
- [Revisiting Random Channel Pruning for Neural Network Compression](https://mlsanity.com/paper/2205.05676/) (2022)
- [Pruning has a disparate impact on model accuracy](https://mlsanity.com/paper/2205.13574/) (2022)
- [Compressing Neural Networks: Towards Determining the Optimal Layer-wise Decomposition](https://mlsanity.com/paper/2107.11442/) (2021)
- [Pruning Neural Networks via Coresets and Convex Geometry: Towards No Assumptions](https://mlsanity.com/paper/2209.08554/) (2022)
- [A Probabilistic Approach to Neural Network Pruning](https://mlsanity.com/paper/2105.10065/) (2021)
- [Why Lottery Ticket Wins? A Theoretical Perspective of Sample Complexity on Pruned Neural Networks](https://mlsanity.com/paper/2110.05667/) (2021)
- [Sparse Flows: Pruning Continuous-depth Models](https://mlsanity.com/paper/2106.12718/) (2021)
- [Model Preserving Compression for Neural Networks](https://mlsanity.com/paper/2108.00065/) (2021)
- [How much pre-training is enough to discover a good subnetwork?](https://mlsanity.com/paper/2108.00259/) (2021)
- [Compress and Compare: Interactively Evaluating Efficiency and Behavior Across ML Model Compression Experiments](https://mlsanity.com/paper/2408.03274/) (2024)
- [One Shot vs. Iterative: Rethinking Pruning Strategies for Model Compression](https://mlsanity.com/paper/2508.13836/) (2025)
- [Compress to Impress: Efficient LLM Adaptation Using a Single Gradient Step on 100 Samples](https://mlsanity.com/paper/2510.20800/) (2025)
- [Non-Convex Optimizations for Machine Learning with Theoretical Guarantee: Robust Matrix Completion and Neural Network Learning](https://mlsanity.com/paper/2306.16557/) (2023)
- [Exploring Vision Neural Network Pruning via Screening Methodology](https://mlsanity.com/paper/2502.07189/) (2025)
- [Theoretical Guarantees for One-Shot Magnitude Pruning and Compute-Adaptive Early Exit](https://mlsanity.com/paper/2609.12337/) (2026)

## Abstract (arXiv)

We introduce a pruning algorithm that provably sparsifies the parameters of a trained model in a way that approximately preserves the model's predictive accuracy. Our algorithm uses a small batch of input points to construct a data-informed importance sampling distribution over the network's parameters, and adaptively mixes a sampling-based and deterministic pruning procedure to discard redundant weights. Our pruning method is simultaneously computationally efficient, provably accurate, and broadly applicable to various network architectures and data distributions. Our empirical comparisons show that our algorithm reliably generates highly compressed networks that incur minimal loss in performance relative to that of the original network. We present experimental results that demonstrate our algorithm's potential to unearth essential network connections that can be trained successfully in isolation, which may be of independent interest.