mlsanityData to 7 Oct 2026

Locally Free Weight Sharing for Network Width Search

arXiv 2102.05258 ↗First posted 10 Feb 2021cs.CVcited by 27

Not on the map. Why?

Abstract (arXiv)

Searching for network width is an effective way to slim deep neural networks with hardware budgets. With this aim, a one-shot supernet is usually leveraged as a performance evaluator to rank the performance wrt~different width. Nevertheless, current methods mainly follow a manually fixed weight sharing pattern, which is limited to distinguish the performance gap of different width. In this paper, to better evaluate each width, we propose a locally free weight sharing strategy (CafeNet) accordingly. In CafeNet, weights are more freely shared, and each width is jointly indicated by its base channels and free channels, where free channels are supposed to loCAte FrEely in a local zone to better represent each width. Besides, we propose to further reduce the search space by leveraging our introduced FLOPs-sensitive bins. As a result, our CafeNet can be trained stochastically and get optimized within a min-min strategy. Extensive experiments on ImageNet, CIFAR-10, CelebA and MS COCO dataset have verified our superiority comparing to other state-of-the-art baselines. For example, our method can further boost the benchmark NAS network EfficientNet-B0 by 0.41% via searching its width more delicately.

Key facts. Cited by 27 papers in the map (data through 7 Oct 2026). Builds on 40 of its 59 references that are in the map. First posted on arXiv 10 Feb 2021.

Builds on

40 of 59 references.

References · 40

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  4. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  5. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  6. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  7. 2016Feature Pyramid Networks for Object Detectioncited by 5,021
  8. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
  9. 2014Deep Learning Face Attributes in the Wildcited by 3,679
  10. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,291
  11. 2018DARTS: Differentiable Architecture Searchcited by 1,782
  12. 2016Pruning Filters for Efficient ConvNetscited by 1,129
28 more
  1. 2018Multi-Task Learning as Multi-Objective Optimizationcited by 809
  2. 2019Once-for-All: Train One Network and Specialize it for Efficient Deploymentcited by 778
  3. 2017Channel Pruning for Accelerating Very Deep Neural Networkscited by 697
  4. 2017Learning Efficient Convolutional Networks through Network Slimmingcited by 673
  5. 2018Rethinking the Value of Network Pruningcited by 595
  6. 2018AMC: AutoML for Model Compression and Acceleration on Mobile Devicescited by 375
  7. 2018Filter Pruning via Geometric Median for Deep Convolutional Neural Networks Accelerationcited by 365
  8. 2018Soft Filter Pruning for Accelerating Deep Convolutional Neural Networkscited by 313
  9. 2019MetaPruning: Meta Learning for Automatic Neural Network Channel Pruningcited by 226
  10. 2018Discrimination-aware Channel Pruning for Deep Neural Networkscited by 171
  11. 2019Towards Optimal Structured CNN Pruning via Generative Adversarial Learningcited by 145
  12. 2018Dynamic Channel Pruning: Feature Boosting and Suppressioncited by 129
  13. 2019Gate Decorator: Global Filter Pruning Method for Accelerating Deep Convolutional Neural Networkscited by 118
  14. 2018Channel Gating Neural Networkscited by 80
  15. 2019Network Pruning via Transformable Architecture Searchcited by 80
  16. 2018AutoPruner: An End-to-End Trainable Filter Pruning Method for Efficient Deep Model Inferencecited by 77
  17. 2019Pruning from Scratchcited by 72
  18. 2020Good Subnetworks Provably Exist: Pruning via Greedy Forward Selectioncited by 70
  19. 2020GreedyNAS: Towards Fast One-Shot NAS with Greedy Supernetcited by 69
  20. 2020DMCP: Differentiable Markov Channel Pruning for Neural Networkscited by 68
  21. 2017More is Less: A More Complicated Network with Less Inference Complexitycited by 66
  22. 2020ISTA-NAS: Efficient and Consistent Neural Architecture Search by Sparse Codingcited by 36
  23. 2019Approximated Oracle Filter Pruning for Destructive CNN Width Optimizationcited by 35
  24. 2021Towards Improving the Consistency, Efficiency, and Flexibility of Differentiable Neural Architecture Searchcited by 27
  25. 2020TF-NAS: Rethinking Three Search Freedoms of Latency-Constrained Differentiable Neural Architecture Searchcited by 21
  26. 2020Data Agnostic Filter Gating for Efficient Deep Networkscited by 7
  27. 2020Network Adjustment: Channel Search Guided by FLOPs Utilization Ratiocited by 4
  28. 2019Bringing Giant Neural Networks Down to Earth with Unlabeled Datacited by 3

Built on by

Cited by year

Citing papers per year: 2021: 8; 2022: 10; 2023: 6; 2024: 2; 2025: 1.

27 papers.

Sort:

  1. 2021Patch Slimming for Efficient Vision Transformerscited by 131
  2. 2021Manifold Regularized Dynamic Network Pruningcited by 43
  3. 2022CHEX: CHannel EXploration for CNN Model Compressioncited by 36
  4. 2023Dynamic Sparse Training with Structured Sparsitycited by 36
  5. 2021Prioritized Architecture Sampling with Monto-Carlo Tree Searchcited by 32
  6. 2021ViTAS: Vision Transformer Architecture Searchcited by 31
  7. 2021SOSP: Efficiently Capturing Global Correlations by Second-Order Structured Pruningcited by 30
  8. 2021BCNet: Searching for Network Width with Bilaterally Coupled Networkcited by 23
  9. 2021K-shot NAS: Learnable Weight-Sharing for NAS with K-shot Supernetscited by 22
  10. 2022On Redundancy and Diversity in Cell-based Neural Architecture Searchcited by 18
  11. 2023Dynamic Sparsity Is Channel-Level Sparsity Learnercited by 17
  12. 2022Trainability Preserving Neural Pruningcited by 16
15 more
  1. 2023Re-mine, Learn and Reason: Exploring the Cross-modal Semantic Correlations for Language-guided HOI detectioncited by 13
  2. 2021GreedyNASv2: Greedier Search with a Greedy Path Filtercited by 12
  3. 2022DyRep: Bootstrapping Training with Dynamic Re-parameterizationcited by 11
  4. 2022HRel: Filter Pruning based on High Relevance between Activation Maps and Class Labelscited by 7
  5. 2022Ensemble Knowledge Guided Sub-network Search and Fine-tuning for Filter Pruningcited by 5
  6. 2024Maxwell's Demon at Work: Efficient Pruning by Leveraging Saturation of Neuronscited by 5
  7. 2022Searching for Network Width with Bilaterally Coupled Networkcited by 4
  8. 2022ScaleNet: Searching for the Model to Scalecited by 4
  9. 2024REPrune: Channel Pruning via Kernel Representative Selectioncited by 4
  10. 2025Adaptive Width Neural Networkscited by 4
  11. 2022Relational Surrogate Loss Learningcited by 2
  12. 2023SUBP: Soft Uniform Block Pruning for 1xN Sparse CNNs Multithreading Accelerationcited by 1
  13. 2022Multi-scale alignment and Spatial ROI Module for COVID-19 Diagnosiscited by 0
  14. 2023Propheter: Prophetic Teacher Guided Long-Tailed Distribution Learningcited by 0
  15. 2023CR-SFP: Learning Consistent Representation for Soft Filter Pruningcited by 0