mlsanityData to 7 Oct 2026

Gator: Customizable Channel Pruning of Neural Networks with Gating

arXiv 2205.15404 ↗First posted 30 May 2022cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

The rise of neural network (NN) applications has prompted an increased interest in compression, with a particular focus on channel pruning, which does not require any additional hardware. Most pruning methods employ either single-layer operations or global schemes to determine which channels to remove followed by fine-tuning of the network. In this paper we present Gator, a channel-pruning method which temporarily adds learned gating mechanisms for pruning of individual channels, and which is trained with an additional auxiliary loss, aimed at reducing the computational cost due to memory, (theoretical) speedup (in terms of FLOPs), and practical, hardware-specific speedup. Gator introduces a new formulation of dependencies between NN layers which, in contrast to most previous methods, enables pruning of non-sequential parts, such as layers on ResNet's highway, and even removing entire ResNet blocks. Gator's pruning for ResNet-50 trained on ImageNet produces state-of-the-art (SOTA) results, such as 50% FLOPs reduction with only 0.4%-drop in top-5 accuracy. Also, Gator outperforms previous pruning models, in terms of GPU latency by running 1.4 times faster. Furthermore, Gator achieves improved top-5 accuracy results, compared to MobileNetV2 and SqueezeNet, for similar runtimes. The source code of this work is available at: https://github.com/EliPassov/gator.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 32 of its 38 references that are in the map. First posted on arXiv 30 May 2022.

Builds on

32 of 38 references.

References · 32

  1. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  2. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  3. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  4. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  5. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
  6. 2017Squeeze-and-Excitation Networkscited by 4,763
  7. 2013Auto-Encoding Variational Bayescited by 3,980
  8. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,708
  9. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,250
  10. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 1,998
  11. 2016Neural Architecture Search with Reinforcement Learningcited by 1,489
  12. 2016Pruning Filters for Efficient ConvNetscited by 1,129
20 more
  1. 2016SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model sizecited by 1,054
  2. 2016XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networkscited by 829
  3. 2017Channel Pruning for Accelerating Very Deep Neural Networkscited by 697
  4. 2020What is the State of Neural Network Pruning?cited by 578
  5. 2018FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Searchcited by 480
  6. 2018AMC: AutoML for Model Compression and Acceleration on Mobile Devicescited by 375
  7. 2018Filter Pruning via Geometric Median for Deep Convolutional Neural Networks Accelerationcited by 365
  8. 2014Speeding up Convolutional Neural Networks with Low Rank Expansionscited by 283
  9. 2017Incremental Network Quantization: Towards Lossless CNNs with Low-Precision Weightscited by 211
  10. 2016Trained Ternary Quantizationcited by 203
  11. 2014Speeding-up Convolutional Neural Networks Using Fine-tuned CP-Decompositioncited by 199
  12. 2018Discrimination-aware Channel Pruning for Deep Neural Networkscited by 171
  13. 2015Compression of Deep Convolutional Neural Networks for Fast and Low Power Mobile Applicationscited by 161
  14. 2015Compressing Neural Networks with the Hashing Trickcited by 150
  15. 2017Convolutional Networks with Adaptive Inference Graphscited by 147
  16. 2018Dynamic Channel Pruning: Feature Boosting and Suppressioncited by 129
  17. 2016Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1cited by 102
  18. 2018Channel Gating Neural Networkscited by 80
  19. 2019Batch-Shaping for Learning Conditional Channel Gated Networkscited by 42
  20. 2018PCAS: Pruning Channels with Attention Statistics for Deep Network Compressioncited by 7

Built on by

Not cited yet.