mlsanityData to 14 Oct 2026

Self-Reorganizing and Rejuvenating CNNs for Increasing Model Capacity Utilization

arXiv 2102.06870 ↗First posted 13 Feb 2021cs.LG cs.CV cs.NEcited by 0

Not on the map. Why?

Abstract (arXiv)

In this paper, we propose self-reorganizing and rejuvenating convolutional neural networks; a biologically inspired method for improving the computational resource utilization of neural networks. The proposed method utilizes the channel activations of a convolution layer in order to reorganize that layers parameters. The reorganized parameters are clustered to avoid parameter redundancies. As such, redundant neurons with similar activations are merged leaving room for the remaining parameters to rejuvenate. The rejuvenated parameters learn different features to supplement those learned by the reorganized surviving parameters. As a result, the network capacity utilization increases improving the baseline network performance without any changes to the network structure. The proposed method can be applied to various network architectures during the training stage, or applied to a pre-trained model improving its performance. Experimental results showed that the proposed method is model-agnostic and can be applied to any backbone architecture increasing its performance due to the elevated utilization of the network capacity.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 29 of its 41 references that are in the map. First posted on arXiv 13 Feb 2021.

Builds on

29 of 41 references.

References · 29

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  4. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  5. 2016Densely Connected Convolutional Networkscited by 6,720
  6. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
  7. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,845
  8. 2015Fast R-CNNcited by 2,933
  9. 2018YOLOv3: An Incremental Improvementcited by 2,659
  10. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 2,031
  11. 2016Neural Architecture Search with Reinforcement Learningcited by 1,499
  12. 2017Accurate, Large Minibatch SGD: Training ImageNet in 1 Hourcited by 1,492
17 more
  1. 2016Pruning Filters for Efficient ConvNetscited by 1,143
  2. 2017Learning Transferable Architectures for Scalable Image Recognitioncited by 1,101
  3. 2017Learning Efficient Convolutional Networks through Network Slimmingcited by 676
  4. 2016DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradientscited by 569
  5. 2019Rigging the Lottery: Making All Tickets Winnerscited by 495
  6. 2017Progressive Neural Architecture Searchcited by 487
  7. 2018Filter Pruning via Geometric Median for Deep Convolutional Neural Networks Accelerationcited by 367
  8. 2016Fully Convolutional Instance-aware Semantic Segmentationcited by 123
  9. 2018Rethinking the Smaller-Norm-Less-Informative Assumption in Channel Pruning of Convolution Layerscited by 112
  10. 2017Evolving Deep Neural Networkscited by 95
  11. 2019Brain-Like Object Recognition with High-Performing Shallow Recurrent ANNscited by 90
  12. 2016DCAN: Deep Contour-Aware Networks for Accurate Gland Segmentationcited by 84
  13. 2015Learning to Segment Object Candidatescited by 79
  14. 2016Learning to Refine Object Segmentscited by 67
  15. 2014DeepID-Net: Deformable Deep Convolutional Neural Networks for Object Detectioncited by 19
  16. 2018Neural Rejuvenation: Improving Deep Network Training by Enhancing Computational Resource Utilizationcited by 10
  17. 2019Training CNNs with Selective Allocation of Channelscited by 5

Built on by

Not cited yet.