mlsanityData to 14 Oct 2026

Pruning Convolutional Filters via Reinforcement Learning with Entropy Minimization

arXiv 2312.04918 ↗First posted 8 Dec 2023cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Structural pruning has become an integral part of neural network optimization, used to achieve architectural configurations which can be deployed and run more efficiently on embedded devices. Previous results showed that pruning is possible with minimum performance loss by utilizing a reinforcement learning agent which makes decisions about the sparsity level of each neural layer by maximizing as a reward the accuracy of the network. We introduce a novel information-theoretic reward function which minimizes the spatial entropy of convolutional activations. This minimization ultimately acts as a proxy for maintaining accuracy, although these two criteria are not related in any way. Our method shows that there is another possibility to preserve accuracy without the need to directly optimize it in the agent's reward function. In our experiments, we were able to reduce the total number of FLOPS of multiple popular neural network architectures by 5-10x, incurring minimal or no performance drop and being on par with the solution found by maximizing the accuracy.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 35 of its 47 references that are in the map. First posted on arXiv 8 Dec 2023.

Builds on

35 of 47 references.

References · 35

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  4. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
  5. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
  6. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,795
  7. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
  8. 2015Continuous control with deep reinforcement learningcited by 3,407
  9. 2020Supervised Contrastive Learningcited by 2,829
  10. 2019RandAugment: Practical automated data augmentation with a reduced search spacecited by 2,269
  11. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,262
  12. 2018DARTS: Differentiable Architecture Searchcited by 1,798
23 more
  1. 2016Neural Architecture Search with Reinforcement Learningcited by 1,499
  2. 2019Self-training with Noisy Student improves ImageNet classificationcited by 1,311
  3. 2018Regularized Evolution for Image Classifier Architecture Searchcited by 909
  4. 2018Efficient Neural Architecture Search via Parameter Sharingcited by 837
  5. 2021A Survey of Quantization Methods for Efficient Neural Network Inferencecited by 815
  6. 2018ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardwarecited by 673
  7. 2018Rethinking the Value of Network Pruningcited by 599
  8. 2020What is the State of Neural Network Pruning?cited by 583
  9. 2019Importance Estimation for Neural Network Pruningcited by 541
  10. 2017Opening the Black Box of Deep Neural Networks via Informationcited by 503
  11. 2019Rigging the Lottery: Making All Tickets Winnerscited by 495
  12. 2019The State of Sparsity in Deep Neural Networkscited by 476
  13. 2019AutoML: A Survey of the State-of-the-Artcited by 390
  14. 2018AMC: AutoML for Model Compression and Acceleration on Mobile Devicescited by 378
  15. 2020Comparing Rewinding and Fine-tuning in Neural Network Pruningcited by 240
  16. 2019Towards moderate overparameterization: global convergence guarantees for training shallow neural networkscited by 176
  17. 2019Population Based Augmentation: Efficient Learning of Augmentation Policy Schedulescited by 154
  18. 2021Convolutional Neural Network Pruning with Structural Redundancy Reductioncited by 73
  19. 2018Automated Machine Learning: From Principles to Practicescited by 62
  20. 2018Learning to Prune Filters in Convolutional Neural Networkscited by 25
  21. 2020Weighted Random Search for Hyperparameter Optimizationcited by 6
  22. 2021Semiotic Aggregation in Deep Learningcited by 4
  23. 2020Weighted Random Search for CNN Hyperparameter Optimizationcited by 3

Built on by

Not cited yet.