Renormalized Sparse Neural Network Pruning
Not on the map. Why?
Abstract (arXiv)
Large neural networks are heavily over-parameterized. This is done because it improves training to optimality. However once the network is trained, this means many parameters can be zeroed, or pruned, leaving an equivalent sparse neural network. We propose renormalizing sparse neural networks in order to improve accuracy. We prove that our method's error converges to zero as network parameters cluster or concentrate. We prove that without renormalizing, the error does not converge to zero in general. We experiment with our method on real world datasets MNIST, Fashion MNIST, and CIFAR-10 and confirm a large improvement in accuracy with renormalization versus standard pruning.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 7 of its 15 references that are in the map. First posted on arXiv 21 Jun 2022.
Builds on
7 of 15 references.
References · 7
- 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,349
- 2017Learning Sparse Neural Networks through L0 Regularizationcited by 583
- 2019The generalization error of random features regression: Precise asymptotics and double descent curvecited by 413
- 2017Exploring the Regularity of Sparse Structure in Convolutional Neural Networkscited by 37
- 2021Generalization Bounds for Sparse Random Feature Expansionscited by 22
- 2022Compression-aware Training of Neural Networks using Frank-Wolfecited by 16
- 2019Stochastic In-Face Frank-Wolfe Methods for Non-Convex Optimization and Sparse Neural Network Trainingcited by 3
Built on by
Not cited yet.