mlsanityData to 14 Oct 2026

EAPruning: Evolutionary Pruning for Vision Transformers and CNNs

arXiv 2210.00181 ↗First posted 1 Oct 2022cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Structured pruning greatly eases the deployment of large neural networks in resource-constrained environments. However, current methods either involve strong domain expertise, require extra hyperparameter tuning, or are restricted only to a specific type of network, which prevents pervasive industrial applications. In this paper, we undertake a simple and effective approach that can be easily applied to both vision transformers and convolutional neural networks. Specifically, we consider pruning as an evolution process of sub-network structures that inherit weights through reconstruction techniques. We achieve a 50% FLOPS reduction for ResNet50 and MobileNetV1, leading to 1.37x and 1.34x speedup respectively. For DeiT-Base, we reach nearly 40% FLOPs reduction and 1.4x speedup. Our code will be made available.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 36 of its 42 references that are in the map. First posted on arXiv 1 Oct 2022.

Builds on

36 of 42 references.

References · 36

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  4. 2020Training data-efficient image transformers & distillation through attentioncited by 4,586
  5. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,845
  6. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,329
  7. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,262
  8. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 2,031
  9. 2019Searching for MobileNetV3cited by 1,994
  10. 2017ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devicescited by 1,200
  11. 2016Pruning Filters for Efficient ConvNetscited by 1,143
  12. 2019Once-for-All: Train One Network and Specialize it for Efficient Deploymentcited by 787
24 more
  1. 2017Channel Pruning for Accelerating Very Deep Neural Networkscited by 702
  2. 2017Learning Efficient Convolutional Networks through Network Slimmingcited by 676
  3. 2018Rethinking the Value of Network Pruningcited by 599
  4. 2016Learning Structured Sparsity in Deep Neural Networkscited by 541
  5. 2019Importance Estimation for Neural Network Pruningcited by 541
  6. 2017ThiNet: A Filter Level Pruning Method for Deep Neural Network Compressioncited by 442
  7. 2018AMC: AutoML for Model Compression and Acceleration on Mobile Devicescited by 378
  8. 2021DeepViT: Towards Deeper Vision Transformercited by 289
  9. 2016Dynamic Network Surgery for Efficient DNNscited by 233
  10. 2019MetaPruning: Meta Learning for Automatic Neural Network Channel Pruningcited by 227
  11. 2016Network Trimming: A Data-Driven Neuron Pruning Approach towards Efficient Deep Architecturescited by 221
  12. 2020BigNAS: Scaling Up Neural Architecture Search with Big Single-Stage Modelscited by 188
  13. 2021AutoFormer: Searching Transformers for Visual Recognitioncited by 188
  14. 2021Learning N:M Fine-grained Structured Sparse Neural Networks From Scratchcited by 179
  15. 2020Soft Threshold Weight Reparameterization for Learnable Sparsitycited by 151
  16. 2018NetAdapt: Platform-Aware Neural Network Adaptation for Mobile Applicationscited by 147
  17. 2020APQ: Joint Search for Network Architecture, Pruning and Quantization Policycited by 99
  18. 2020ResRep: Lossless CNN Pruning via Decoupling Remembering and Forgettingcited by 85
  19. 2021ResT: An Efficient Transformer for Visual Recognitioncited by 79
  20. 2021Only Train Once: A One-Shot Neural Network Training And Pruning Frameworkcited by 70
  21. 2017NeST: A Neural Network Synthesis Tool Based on a Grow-and-Prune Paradigmcited by 60
  22. 2021A Unified Pruning Framework for Vision Transformerscited by 40
  23. 2022N2NSkip: Learning Highly Sparse Networks using Neuron-to-Neuron Skip Connectionscited by 9
  24. 2017Towards Evolutional Compressioncited by 1

Built on by

Not cited yet.