mlsanityData to 7 Oct 2026

Data-free parameter pruning for Deep Neural Networks

arXiv 1507.06149 ↗First posted 22 Jul 2015cs.CVcited by 112

Not on the map. Why?

Abstract (arXiv)

Deep Neural nets (NNs) with millions of parameters are at the heart of many state-of-the-art computer vision systems today. However, recent works have shown that much smaller models can achieve similar levels of performance. In this work, we address the problem of pruning parameters in a trained NN model. Instead of removing individual weights one at a time as done in previous works, we remove one neuron at a time. We show how similar neurons are redundant, and propose a systematic way to remove them. Our experiments in pruning the densely connected layers show that we can remove upto 85% of the total parameters in an MNIST-trained network, and about 35% for AlexNet without significantly affecting performance. Our method can be applied on top of most networks with a fully connected layer to give a smaller network.

Key facts. Cited by 112 papers in the map (data through 7 Oct 2026). First posted on arXiv 22 Jul 2015.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 26; 2022: 24; 2023: 25; 2024: 21; 2025: 10; 2026: 6.

112 papers.

Sort:

By area:

  1. 2023LLM-Pruner: On the Structural Pruning of Large Language Modelscited by 643
  2. 2022Not All Patches are What You Need: Expediting Vision Transformers via Token Reorganizationscited by 359
  3. 2021Pruning and Quantization for Deep Neural Network Acceleration: A Surveycited by 259
  4. 2024The Unreasonable Ineffectiveness of the Deeper Layerscited by 219
  5. 2022A Fast Post-Training Pruning Framework for Transformerscited by 161
  6. 2022Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networkscited by 126
  7. 2022A Comprehensive Survey on Model Quantization for Deep Neural Networks in Image Classificationcited by 64
  8. 2021CheXtransfer: Performance and Parameter Efficiency of ImageNet Models for Chest X-Ray Interpretationcited by 57
  9. 2021A Survey on Green Deep Learningcited by 52
  10. 2022Online Continual Learning for Embedded Devicescited by 48
  11. 2021Zero-shot Adversarial Quantizationcited by 44
  12. 2022Blueprint Separable Residual Network for Efficient Image Super-Resolutioncited by 44
38 more
  1. 2022The Combinatorial Brain Surgeon: Pruning Weights That Cancel One Another in Neural Networkscited by 41
  2. 2024Elucidating the Design Space of Dataset Condensationcited by 40
  3. 2024Pruner-Zero: Evolving Symbolic Pruning Metric from scratch for Large Language Modelscited by 40
  4. 2024Attention-Driven Training-Free Efficiency Enhancement of Diffusion Modelscited by 37
  5. 2022Enable Deep Learning on Mobile Devices: Methods, Systems, and Applicationscited by 32
  6. 2023What Matters In The Structured Pruning of Generative Language Models?cited by 30
  7. 2021Post-training deep neural network pruning via layer-wise calibrationcited by 23
  8. 2022Data-Efficient Structured Pruning via Submodular Optimizationcited by 22
  9. 2023Unified Data-Free Compression: Pruning and Quantization without Fine-Tuningcited by 22
  10. 2024DELT: A Simple Diversity-driven EarlyLate Training for Dataset Distillationcited by 20
  11. 2021Scaling Up Exact Neural Network Compression by ReLU Stabilitycited by 19
  12. 2021RED : Looking for Redundancies for Data-Free Structured Compression of Deep Neural Networkscited by 19
  13. 2021RED++ : Data-Free Pruning of Deep Neural Networks via Input Splitting and Output Mergingcited by 19
  14. 2022OPQ: Compressing Deep Neural Networks with One-shot Pruning-Quantizationcited by 19
  15. 2022NEVIS'22: A Stream of 100 Tasks Sampled from 30 Years of Computer Vision Researchcited by 18
  16. 2021Stealthy Backdoors as Compression Artifactscited by 17
  17. 2021Truly Sparse Neural Networks at Scalecited by 15
  18. 2021A Probabilistic Approach to Neural Network Pruningcited by 15
  19. 2021Why Lottery Ticket Wins? A Theoretical Perspective of Sample Complexity on Pruned Neural Networkscited by 14
  20. 2022HFedMS: Heterogeneous Federated Learning with Memorable Data Semantics in Industrial Metaversecited by 14
  21. 2022Is Complexity Required for Neural Network Pruning? A Case Study on Global Magnitude Pruningcited by 13
  22. 2024Self-supervised Dataset Distillation: A Good Compression Is All You Needcited by 13
  23. 2024From Efficient Multimodal Models to World Models: A Surveycited by 13
  24. 2024Hadamard Adapter: An Extreme Parameter-Efficient Adapter Tuning Method for Pre-trained Language Modelscited by 13
  25. 2023Going Beyond Neural Network Feature Similarity: The Network Feature Complexity and Its Interpretation Using Category Theorycited by 12
  26. 2024Mitigating Selection Bias with Node Pruning and Auxiliary Optionscited by 12
  27. 2023Accurate Retraining-free Pruning for Pretrained Encoder-based Language Modelscited by 11
  28. 2024Expanding Sparse Tuning for Low Memory Usagecited by 10
  29. 2021Resource-Efficient Deep Learning: A Survey on Model-, Arithmetic-, and Implementation-Level Techniquescited by 9
  30. 2022SInGE: Sparsity via Integrated Gradients Estimation of Neuron Relevancecited by 9
  31. 2023Data-Free Quantization via Mixed-Precision Compensation without Fine-Tuningcited by 9
  32. 2023Efficient Neural Networks for Tiny Machine Learning: A Comprehensive Reviewcited by 9
  33. 2023Enabling Resource-efficient AIoT System with Cross-level Optimization: A surveycited by 8
  34. 2024Structurally Prune Anything: Any Architecture, Any Framework, Any Timecited by 8
  35. 2023Accelerating Distributed ML Training via Selective Synchronizationcited by 7
  36. 2023On the Opportunities of Green Computing: A Surveycited by 7
  37. 2024Hierarchical Skip Decoding for Efficient Autoregressive Text Generationcited by 7
  38. 2022Developmental Plasticity-inspired Adaptive Pruning for Deep Spiking and Artificial Neural Networkscited by 6

50 most cited of 112 All 112 (JSON)