mlsanityData to 14 Oct 2026

Tiered Pruning for Efficient Differentialble Inference-Aware Neural Architecture Search

arXiv 2209.11785 ↗First posted 23 Sep 2022cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

We propose three novel pruning techniques to improve the cost and results of inference-aware Differentiable Neural Architecture Search (DNAS). First, we introduce Prunode, a stochastic bi-path building block for DNAS, which can search over inner hidden dimensions with O(1) memory and compute complexity. Second, we present an algorithm for pruning blocks within a stochastic layer of the SuperNet during the search. Third, we describe a novel technique for pruning unnecessary stochastic layers during the search. The optimized models resulting from the search are called PruNet and establishes a new state-of-the-art Pareto frontier for NVIDIA V100 in terms of inference latency for ImageNet Top-1 image classification accuracy. PruNet as a backbone also outperforms GPUNet and EfficientNet on the COCO object detection task on inference latency relative to mean Average Precision (mAP).

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 45 of its 59 references that are in the map. First posted on arXiv 23 Sep 2022.

Builds on

45 of 59 references.

References · 45

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  4. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  5. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
  6. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,795
  7. 2017Squeeze-and-Excitation Networkscited by 4,789
  8. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,845
  9. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,739
  10. 2018DARTS: Differentiable Architecture Searchcited by 1,798
  11. 2021EfficientNetV2: Smaller Models and Faster Trainingcited by 1,135
  12. 2019EfficientDet: Scalable and Efficient Object Detectioncited by 1,020
33 more
  1. 2020Designing Network Design Spacescited by 960
  2. 2018MnasNet: Platform-Aware Neural Architecture Search for Mobilecited by 943
  3. 2018Efficient Neural Architecture Search via Parameter Sharingcited by 837
  4. 2019Once-for-All: Train One Network and Specialize it for Efficient Deploymentcited by 787
  5. 2018AutoAugment: Learning Augmentation Policies from Datacited by 730
  6. 2018ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardwarecited by 673
  7. 2017Progressive Neural Architecture Searchcited by 487
  8. 2018FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Searchcited by 485
  9. 2018BOHB: Robust and Efficient Hyperparameter Optimization at Scalecited by 470
  10. 2018SNAS: Stochastic Neural Architecture Searchcited by 289
  11. 2019Progressive Differentiable Architecture Search: Bridging the Depth Gap between Search and Evaluationcited by 230
  12. 2016FractalNet: Ultra-Deep Neural Networks without Residualscited by 216
  13. 2018A System for Massively Parallel Hyperparameter Tuningcited by 202
  14. 2019Understanding and Robustifying Differentiable Architecture Searchcited by 178
  15. 2018Auto-Keras: An Efficient Neural Architecture Search Systemcited by 171
  16. 2018Neural Architecture Optimizationcited by 167
  17. 2020FBNetV2: Differentiable Neural Architecture Search for Spatial and Channel Dimensionscited by 152
  18. 2018NetAdapt: Platform-Aware Neural Network Adaptation for Mobile Applicationscited by 147
  19. 2019DARTS+: Improved Differentiable Architecture Search with Early Stoppingcited by 105
  20. 2018ChamNet: Towards Efficient Network Design through Platform-Aware Model Adaptationcited by 76
  21. 2020Few-shot Neural Architecture Searchcited by 50
  22. 2018Path-Level Network Transformation for Efficient Architecture Searchcited by 44
  23. 2018DPP-Net: Device-aware Progressive Search for Pareto-optimal Neural Architecturescited by 40
  24. 2019AtomNAS: Fine-Grained End-to-End Neural Architecture Searchcited by 40
  25. 2019Improving One-shot NAS by Suppressing the Posterior Fadingcited by 28
  26. 2020Evolving Search Space for Neural Architecture Searchcited by 23
  27. 2021Searching for Fast Model Families on Datacenter Acceleratorscited by 22
  28. 2019UNAS: Differentiable Architecture Search Meets Reinforcement Learningcited by 21
  29. 2020Neural Architecture Design for GPU-Efficient Networkscited by 13
  30. 2020Progressive Automatic Design of Search Space for One-Shot Neural Architecture Searchcited by 10
  31. 2021HALP: Hardware-Aware Latency Pruningcited by 8
  32. 2022GPUNet: Searching the Deployable Convolution Neural Networks for GPUscited by 7
  33. 2021An Analysis of Super-Net Heuristics in Weight-Sharing NAScited by 2

Built on by

Not cited yet.