mlsanityData to 7 Oct 2026

DropNet: Reducing Neural Network Complexity via Iterative Pruning

arXiv 2207.06646 ↗First posted 14 Jul 2022cs.LG cs.AIcited by 29

Not on the map. Why?

Abstract (arXiv)

Modern deep neural networks require a significant amount of computing time and power to train and deploy, which limits their usage on edge devices. Inspired by the iterative weight pruning in the Lottery Ticket Hypothesis, we propose DropNet, an iterative pruning method which prunes nodes/filters to reduce network complexity. DropNet iteratively removes nodes/filters with the lowest average post-activation value across all training samples. Empirically, we show that DropNet is robust across diverse scenarios, including MLPs and CNNs using the MNIST, CIFAR-10 and Tiny ImageNet datasets. We show that up to 90% of the nodes/filters can be removed without any significant loss of accuracy. The final pruned network performs well even with reinitialization of the weights and biases. DropNet also has similar accuracy to an oracle which greedily removes nodes/filters one at a time to minimise training loss, highlighting its effectiveness.

Key facts. Cited by 29 papers in the map (data through 7 Oct 2026). Builds on 15 of its 38 references that are in the map. First posted on arXiv 14 Jul 2022.

Builds on

15 of 38 references.

References · 15

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  4. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  5. 2014Going Deeper with Convolutionscited by 4,231
  6. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,250
  7. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 1,998
  8. 2016Pruning Filters for Efficient ConvNetscited by 1,129
  9. 2017Channel Pruning for Accelerating Very Deep Neural Networkscited by 697
  10. 2017Learning Efficient Convolutional Networks through Network Slimmingcited by 673
  11. 2018Rethinking the Value of Network Pruningcited by 595
  12. 2016Learning Structured Sparsity in Deep Neural Networkscited by 538
3 more
  1. 2015Deep Learning with Limited Numerical Precisioncited by 368
  2. 2015Structured Pruning of Deep Convolutional Neural Networkscited by 148
  3. 2016Learning the Number of Neurons in Deep Networkscited by 69

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 3; 2023: 10; 2024: 7; 2025: 7; 2026: 2.

29 papers.

Sort:

By area:

  1. 2023Structured Pruning for Deep Convolutional Neural Networks: A surveycited by 122
  2. 2023Fluctuation-based Adaptive Structured Pruning for Large Language Modelscited by 110
  3. 2025Empowering Edge Intelligence: A Comprehensive Survey on On-Device AI Modelscited by 44
  4. 2024Reassessing Layer Pruning in LLMs: New Insights and Methodscited by 23
  5. 2024Pruning for Robust Concept Erasing in Diffusion Modelscited by 22
  6. 2023A Survey on Approximate Edge AI for Energy Efficient Autonomous Driving Servicescited by 14
  7. 2023Relationship between Model Compression and Adversarial Robustness: A Review of Current Evidencecited by 6
  8. 2025FastFLUX: Pruning FLUX with Block-wise Replacement and Sandwich Trainingcited by 5
  9. 2023When Layers Play the Lottery, all Tickets Win at Initializationcited by 3
  10. 2023Approximate Computing Survey, Part II: Application-Specific & Architectural Approximation Techniques and Applicationscited by 3
  11. 2023Benchmarking Adversarial Robustness of Compressed Deep Learning Modelscited by 3
  12. 2022Neural Networks Reduction via Lumpingcited by 2
17 more
  1. 2022COST-EFF: Collaborative Optimization of Spatial and Temporal Efficiency with Slenderized Multi-exit Language Modelscited by 2
  2. 2023Mixture-of-Linguistic-Experts Adapters for Improving and Interpreting Pre-trained Language Modelscited by 2
  3. 2025Lightweight Embedded FPGA Deployment of Learned Image Compression with Knowledge Distillation and Hybrid Quantizationcited by 2
  4. 2025An Efficient Training Algorithm for Models with Block-wise Sparsitycited by 2
  5. 2025Hyperpruning: Efficient Search through Pruned Variants of Recurrent Neural Networks Leveraging Lyapunov Spectrumcited by 1
  6. 2022Optimizing Learning Rate Schedules for Iterative Pruning of Deep Neural Networkscited by 0
  7. 2023Understanding Activation Patterns in Artificial Neural Networks by Exploring Stochastic Processescited by 0
  8. 2023Equitable-FL: Federated Learning with Sparsity for Resource-Constrained Environmentcited by 0
  9. 2024Dependable Distributed Training of Compressed Machine Learning Modelscited by 0
  10. 2024Advancing IIoT with Over-the-Air Federated Learning: The Role of Iterative Magnitude Pruningcited by 0
  11. 2024HASS: Hardware-Aware Sparsity Search for Dataflow DNN Acceleratorcited by 0
  12. 2024A Comprehensive Study of Structural Pruning for Vision Modelscited by 0
  13. 2024FedTSA: A Cluster-based Two-Stage Aggregation Method for Model-heterogeneous Federated Learningcited by 0
  14. 2025Exploring Vision Neural Network Pruning via Screening Methodologycited by 0
  15. 2025Vanishing Contributions: A Unified Framework for Smooth and Iterative Model Compressioncited by 0
  16. 2026Regularisation in neural networks: a survey and empirical analysis of approachescited by 0
  17. 2026S4oP: Operator-level Pruning of Structured State Space Models for Resource-Constrained Devicescited by 0