mlsanityData to 7 Oct 2026

Single Shot Structured Pruning Before Training

arXiv 2007.00389 ↗First posted 1 Jul 2020cs.LG stat.MLcited by 19

Not on the map. Why?

Abstract (arXiv)

We introduce a method to speed up training by 2x and inference by 3x in deep neural networks using structured pruning applied before training. Unlike previous works on pruning before training which prune individual weights, our work develops a methodology to remove entire channels and hidden units with the explicit aim of speeding up training and inference. We introduce a compute-aware scoring mechanism which enables pruning in units of sensitivity per FLOP removed, allowing even greater speed ups. Our method is fast, easy to implement, and needs just one forward/backward pass on a single batch of data to complete pruning before training begins.

Key facts. Cited by 19 papers in the map (data through 7 Oct 2026). First posted on arXiv 1 Jul 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 4; 2022: 4; 2023: 4; 2024: 2; 2025: 3; 2026: 2.

19 papers.

Sort:

  1. 2023A Survey on Deep Neural Network Pruning-Taxonomy, Comparison, Analysis, and Recommendationscited by 200
  2. 2021MEST: Accurate and Fast Memory-Economic Sparse Training Framework on the Edgecited by 77
  3. 2022Advancing Model Pruning via Bi-level Optimizationcited by 55
  4. 2022Prospect Pruning: Finding Trainable Weights at Initialization using Meta-Gradientscited by 34
  5. 2021SOSP: Efficiently Capturing Global Correlations by Second-Order Structured Pruningcited by 30
  6. 2021Effective Model Sparsification by Scheduled Grow-and-Prune Methodscited by 25
  7. 2021Differentiable Network Pruning for Microcontrollerscited by 15
  8. 2022Layer Freezing & Data Sieving: Missing Pieces of a Generic Framework for Sparse Trainingcited by 11
  9. 2024EPSD: Early Pruning with Self-Distillation for Efficient Model Compressioncited by 6
  10. 2023Sparse then Prune: Toward Efficient Vision Transformerscited by 4
  11. 2023AUTOSPARSE: Towards Automated Sparse Training of Deep Neural Networkscited by 2
  12. 2023Pruning at Initialization -- A Sketching Perspectivecited by 2
7 more
  1. 2025Pruning-based Data Selection and Network Fusion for Efficient Deep Learningcited by 2
  2. 2024Rapid Deployment of DNNs for Edge Computing via Structured Pruning at Initializationcited by 1
  3. 2025LinDeps: A Fine-tuning Free Post-Pruning Method to Remove Layer-Wise Linear Dependencies with Guaranteed Performance Preservationcited by 1
  4. 2022Hyperparameter Optimization with Neural Network Pruningcited by 0
  5. 2025Exploring Vision Neural Network Pruning via Screening Methodologycited by 0
  6. 2026PruneFuse: Efficient Data Selection via Weight Pruning and Network Fusioncited by 0
  7. 2026FedASAP: Activation Statistics-driven Structured Adaptive Pruning for Efficient Personalized Federated Learning for Lesion Segmentation on brain MRIcited by 0