Deadwooding: Robust Global Pruning for Deep Neural Networks
Not on the map. Why?
Abstract (arXiv)
The ability of Deep Neural Networks to approximate highly complex functions is key to their success. This benefit, however, comes at the expense of a large model size, which challenges its deployment in resource-constrained environments. Pruning is an effective technique used to limit this issue, but often comes at the cost of reduced accuracy and adversarial robustness. This paper addresses these shortcomings and introduces Deadwooding, a novel global pruning technique that exploits a Lagrangian Dual method to encourage model sparsity while retaining accuracy and ensuring robustness. The resulting model is shown to significantly outperform the state-of-the-art studies in measures of robustness and accuracy.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 34 of its 62 references that are in the map. First posted on arXiv 10 Feb 2022.
Builds on
34 of 62 references.
References · 34
- 2015Distilling the Knowledge in a Neural Networkcited by 10,324
- 2014Explaining and Harnessing Adversarial Examplescited by 7,364
- 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
- 2013Intriguing properties of neural networkscited by 5,155
- 2019Optuna: A Next-generation Hyperparameter Optimization Frameworkcited by 2,501
- 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,329
- 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,262
- 2015Learning both Weights and Connections for Efficient Neural Networkscited by 2,031
- 2020Knowledge Distillation: A Surveycited by 1,603
- 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,451
- 2015DeepFool: a simple and accurate method to fool deep neural networkscited by 1,318
- 2016Pruning Filters for Efficient ConvNetscited by 1,143
22 more
- 2020Fast is better than free: Revisiting adversarial trainingcited by 727
- 2015Distillation as a Defense to Adversarial Perturbations against Deep Neural Networkscited by 678
- 2017To prune, or not to prune: exploring the efficacy of pruning for model compressioncited by 603
- 2020What is the State of Neural Network Pruning?cited by 583
- 2018Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Surveycited by 460
- 2019On Evaluating Adversarial Robustnesscited by 447
- 2020Movement Pruning: Adaptive Sparsity by Fine-Tuningcited by 403
- 2019Minimally distorted Adversarial Examples with a Fast Adaptive Boundary Attackcited by 334
- 2021Pre-Trained Models: Past, Present and Futurecited by 319
- 2018Model compression via distillation and quantizationcited by 289
- 2018Adversarial Attacks and Defences: A Surveycited by 259
- 2019What's Hidden in a Randomly Weighted Neural Network?cited by 237
- 2018A Systematic DNN Weight Pruning Framework using Alternating Direction Method of Multiplierscited by 120
- 2021Do We Actually Need Dense Over-Parameterization? In-Time Over-Parameterization in Sparse Trainingcited by 110
- 2019Winning the Lottery with Continuous Sparsificationcited by 100
- 2018Sparse DNNs with Improved Adversarial Robustnesscited by 72
- 2019The Generalization-Stability Tradeoff In Neural Network Pruningcited by 54
- 2018Deep Defense: Training DNNs with Improved Adversarial Robustnesscited by 42
- 2016Deep Model Compression: Distilling Knowledge from Noisy Teacherscited by 36
- 2018Adversarial Risk and Robustness: General Definitions and Implications for the Uniform Distributioncited by 32
- 2022On the Real-World Adversarial Robustness of Real-Time Semantic Segmentation Models for Autonomous Drivingcited by 30
- 2019Towards Compact and Robust Deep Neural Networkscited by 23
Built on by
Citing papers per year: 2024: 2.
2 papers.