One-shot Network Pruning at Initialization with Discriminative Image Patches
Not on the map. Why?
Abstract (arXiv)
One-shot Network Pruning at Initialization (OPaI) is an effective method to decrease network pruning costs. Recently, there is a growing belief that data is unnecessary in OPaI. However, we obtain an opposite conclusion by ablation experiments in two representative OPaI methods, SNIP and GraSP. Specifically, we find that informative data is crucial to enhancing pruning performance. In this paper, we propose two novel methods, Discriminative One-shot Network Pruning (DOP) and Super Stitching, to prune the network by high-level visual discriminative image patches. Our contributions are as follows. (1) Extensive experiments reveal that OPaI is data-dependent. (2) Super Stitching performs significantly better than the original OPaI method on benchmark ImageNet, especially in a highly compressed model.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 33 of its 43 references that are in the map. First posted on arXiv 13 Sep 2022.
Builds on
33 of 43 references.
References · 33
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,250
- 2015Learning both Weights and Connections for Efficient Neural Networkscited by 1,998
- 2017Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)cited by 1,309
- 2018SNIP: Single-shot Network Pruning based on Connection Sensitivitycited by 802
- 2018Rethinking the Value of Network Pruningcited by 595
- 2020What is the State of Neural Network Pruning?cited by 578
- 2021Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networkscited by 521
- 2020Pruning neural networks without any data by iteratively conserving synaptic flowcited by 475
- 2019The State of Sparsity in Deep Neural Networkscited by 471
- 2020Picking Winning Tickets Before Training by Preserving Gradient Flowcited by 467
- 2017Efficient Processing of Deep Neural Networks: A Tutorial and Surveycited by 427
21 more
- 2019On Completeness-aware Concept-Based Explanations in Deep Neural Networkscited by 239
- 2019Deconstructing Lottery Tickets: Zeros, Signs, and the Supermaskcited by 206
- 2020Proving the Lottery Ticket Hypothesis: Pruning is All You Needcited by 200
- 2020Pruning Neural Networks at Initialization: Why are We Missing the Mark?cited by 181
- 2019Drawing Early-Bird Tickets: Towards More Efficient Training of Deep Networkscited by 160
- 2020Sparse GPU Kernels for Deep Learningcited by 138
- 2019One ticket to win them all: generalizing lottery ticket initializations across datasets and optimizerscited by 102
- 2020The Lottery Tickets Hypothesis for Supervised and Self-supervised Pre-training in Computer Vision Modelscited by 91
- 2021Only Train Once: A One-Shot Neural Network Training And Pruning Frameworkcited by 70
- 2020Sanity-Checking Pruning Methods: Random Tickets can Win the Jackpotcited by 66
- 2020Progressive Skeletonization: Trimming more fat from a network at initializationcited by 64
- 2021Recent Advances on Neural Network Pruning at Initializationcited by 41
- 2019Data-Independent Neural Pruning via Coresetscited by 35
- 2020A Gradient Flow Framework For Analyzing Network Pruningcited by 35
- 2022Prospect Pruning: Finding Trainable Weights at Initialization using Meta-Gradientscited by 34
- 2020Robust Pruning at Initializationcited by 31
- 2019One-Shot Pruning of Recurrent Neural Networks by Jacobian Spectrum Evaluationcited by 14
- 2021Connectivity Matters: Neural Network Pruning Through the Lens of Effective Sparsitycited by 14
- 2022Dimensionality Reduced Training by Pruning and Freezing Parts of a Deep Neural Network, a Surveycited by 12
- 2022N2NSkip: Learning Highly Sparse Networks using Neuron-to-Neuron Skip Connectionscited by 9
- 2021COPS: Controlled Pruning Before Training Startscited by 6
Built on by
Citing papers per year: 2023: 1; 2024: 1.
2 papers.