mlsanityData to 7 Oct 2026

Fast ConvNets Using Group-wise Brain Damage

arXiv 1506.02515 ↗First posted 8 Jun 2015cs.CVcited by 81

Not on the map. Why?

Abstract (arXiv)

We revisit the idea of brain damage, i.e. the pruning of the coefficients of a neural network, and suggest how brain damage can be modified and used to speedup convolutional layers. The approach uses the fact that many efficient implementations reduce generalized convolutions to matrix multiplications. The suggested brain damage process prunes the convolutional kernel tensor in a group-wise fashion by adding group-sparsity regularization to the standard training process. After such group-wise pruning, convolutions can be reduced to multiplications of thinned dense matrices, which leads to speedup. In the comparison on AlexNet, the method achieves very competitive performance.

Key facts. Cited by 81 papers in the map (data through 7 Oct 2026). First posted on arXiv 8 Jun 2015.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 25; 2022: 24; 2023: 16; 2024: 11; 2025: 4; 2026: 1.

81 papers.

Sort:

  1. 2021Pruning and Quantization for Deep Neural Network Acceleration: A Surveycited by 259
  2. 2022Beyond Transmitting Bits: Context, Semantics, and Task-Oriented Communicationscited by 166
  3. 2021Patch Slimming for Efficient Vision Transformerscited by 131
  4. 2021Group Fisher Pruning for Practical Network Compressioncited by 130
  5. 2021Distributed Learning in Wireless Networks: Recent Progress and Future Challengescited by 125
  6. 2023Structured Pruning for Deep Convolutional Neural Networks: A surveycited by 122
  7. 2021CDFI: Compression-Driven Network Design for Frame Interpolationcited by 55
  8. 2022Blueprint Separable Residual Network for Efficient Image Super-Resolutioncited by 44
  9. 2022The Combinatorial Brain Surgeon: Pruning Weights That Cancel One Another in Neural Networkscited by 41
  10. 2022Enable Deep Learning on Mobile Devices: Methods, Systems, and Applicationscited by 32
  11. 2022Pruning's Effect on Generalization Through the Lens of Training and Regularizationcited by 31
  12. 2021Distribution-sensitive Information Retention for Accurate Binary Neural Networkcited by 25
38 more
  1. 2021TransTailor: Pruning the Pre-trained Model for Improved Transfer Learningcited by 24
  2. 2023BiBench: Benchmarking and Analyzing Network Binarizationcited by 22
  3. 2023Unified Data-Free Compression: Pruning and Quantization without Fine-Tuningcited by 22
  4. 2022Pruning Neural Networks via Coresets and Convex Geometry: Towards No Assumptionscited by 20
  5. 2021Scaling Up Exact Neural Network Compression by ReLU Stabilitycited by 19
  6. 2021Double Similarity Distillation for Semantic Image Segmentationcited by 19
  7. 2024ALPS: Improved Optimization for Highly Sparse One-Shot Pruning for Large Language Modelscited by 18
  8. 2024OSSCAR: One-Shot Structured Pruning in Vision and Language Models with Combinatorial Optimizationcited by 17
  9. 2021Neural Network Training Using ℓ1-Regularization and Bi-fidelity Datacited by 16
  10. 2022Trainability Preserving Neural Pruningcited by 16
  11. 2022Continual Prune-and-Select: Class-incremental learning with specialized subnetworkscited by 16
  12. 2024LSK3DNet: Towards Effective and Efficient 3D Perception with Large Sparse Kernelscited by 15
  13. 2021Connectivity Matters: Neural Network Pruning Through the Lens of Effective Sparsitycited by 14
  14. 2022Provable and Efficient Continual Representation Learningcited by 14
  15. 2021SeReNe: Sensitivity based Regularization of Neurons for Structured Sparsity in Neural Networkscited by 12
  16. 2021LANA: Latency Aware Network Accelerationcited by 11
  17. 2021Neural network relief: a pruning algorithm based on neural activitycited by 10
  18. 2022Recall Distortion in Neural Network Pruning and the Undecayed Pruning Algorithmcited by 9
  19. 2022Sequential Attention for Feature Selectioncited by 9
  20. 2022Knowledge Distillation Meets Open-Set Semi-Supervised Learningcited by 8
  21. 2023Getting Away with More Network Pruning: From Sparsity to Geometry and Linear Regionscited by 8
  22. 2021Deep Model Compression based on the Training Historycited by 7
  23. 2021Learning on Hardware: A Tutorial on Neural Network Accelerators and Co-Processorscited by 6
  24. 2022Sparsity-guided Network Design for Frame Interpolationcited by 6
  25. 2024Data-independent Module-aware Pruning for Hierarchical Vision Transformerscited by 6
  26. 2023Low Rank Optimization for Efficient Deep Learning: Making A Balance between Compact Architecture and Fast Trainingcited by 5
  27. 2024Maxwell's Demon at Work: Efficient Pruning by Leveraging Saturation of Neuronscited by 5
  28. 2021FedSkel: Efficient Federated Learning on Heterogeneous Systems with Skeleton Gradients Updatecited by 4
  29. 2022Auto-Compressing Subset Pruning for Semantic Image Segmentationcited by 4
  30. 2022Fast Lossless Neural Compression with Integer-Only Discrete Flowscited by 4
  31. 2023Riemannian Low-Rank Model Compression for Federated Learning with Over-the-Air Aggregationcited by 4
  32. 2024Neuralink: Fast LLM Inference on Smartphones with Neuron Co-Activation Linkingcited by 4
  33. 2021High Performance Convolution Using Sparsity and Patterns for Inference in Deep Convolutional Neural Networkscited by 3
  34. 2021Alternate Model Growth and Pruning for Efficient Training of Recommendation Systemscited by 3
  35. 2021Deep Neural Compression Via Concurrent Pruning and Self-Distillationcited by 3
  36. 2022Canonical convolutional neural networkscited by 3
  37. 2023Approximate Computing Survey, Part II: Application-Specific & Architectural Approximation Techniques and Applicationscited by 3
  38. 2024T3DNet: Compressing Point Cloud Models for Lightweight 3D Recognitioncited by 3

50 most cited of 81 All 81 (JSON)