mlsanityData to 7 Oct 2026

GPUNet: Searching the Deployable Convolution Neural Networks for GPUs

arXiv 2205.00841 ↗First posted 26 Apr 2022cs.CV cs.AIcited by 7

Not on the map. Why?

Abstract (arXiv)

Customizing Convolution Neural Networks (CNN) for production use has been a challenging task for DL practitioners. This paper intends to expedite the model customization with a model hub that contains the optimized models tiered by their inference latency using Neural Architecture Search (NAS). To achieve this goal, we build a distributed NAS system to search on a novel search space that consists of prominent factors to impact latency and accuracy. Since we target GPU, we name the NAS optimized models as GPUNet, which establishes a new SOTA Pareto frontier in inference latency and accuracy. Within 1ms, GPUNet is 2x faster than EfficientNet-X and FBNetV3 with even better accuracy. We also validate GPUNet on detection tasks, and GPUNet consistently outperforms EfficientNet-X and FBNetV3 on COCO detection tasks in both latency and accuracy. All of these data validate that our NAS system is effective and generic to handle different design tasks. With this NAS system, we expand GPUNet to cover a wide range of latency targets such that DL practitioners can deploy our models directly in different scenarios.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 32 of its 43 references that are in the map. First posted on arXiv 26 Apr 2022.

Builds on

32 of 43 references.

References · 32

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  4. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  5. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  6. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
  7. 2017Squeeze-and-Excitation Networkscited by 4,763
  8. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,827
  9. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,590
  10. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,360
  11. 2019Searching for MobileNetV3cited by 1,984
  12. 2019Self-training with Noisy Student improves ImageNet classificationcited by 1,304
20 more
  1. 2017Cascade R-CNN: Delving into High Quality Object Detectioncited by 1,275
  2. 2021EfficientNetV2: Smaller Models and Faster Trainingcited by 1,127
  3. 2017Learning Transferable Architectures for Scalable Image Recognitioncited by 1,097
  4. 2020Designing Network Design Spacescited by 954
  5. 2018Efficient Neural Architecture Search via Parameter Sharingcited by 831
  6. 2018ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardwarecited by 666
  7. 2018FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Searchcited by 480
  8. 2020ResNeSt: Split-Attention Networkscited by 445
  9. 2021Pay Attention to MLPscited by 367
  10. 2017The Expressive Power of Neural Networks: A View from the Widthcited by 318
  11. 2019Auto-DeepLab: Hierarchical Neural Architecture Search for Semantic Image Segmentationcited by 258
  12. 2020BigNAS: Scaling Up Neural Architecture Search with Big Single-Stage Modelscited by 187
  13. 2020FBNetV2: Differentiable Neural Architecture Search for Spatial and Channel Dimensionscited by 152
  14. 2019Evaluating the Search Phase of Neural Architecture Searchcited by 120
  15. 2020TResNet: High Performance GPU-Dedicated Architecturecited by 119
  16. 2020Learning Search Space Partition for Black-box Optimization using Monte Carlo Tree Searchcited by 87
  17. 2020Few-shot Neural Architecture Searchcited by 50
  18. 2019AlphaX: eXploring Neural Architectures with Deep Neural Networks and Monte Carlo Tree Searchcited by 33
  19. 2021Searching for Fast Model Families on Datacenter Acceleratorscited by 22
  20. 2020How to Train Your Super-Net: An Analysis of Training Heuristics in Weight-Sharing NAScited by 17

Built on by

Cited by year

Citing papers per year: 2022: 2; 2023: 2; 2024: 3.

7 papers.

  1. 2023Knowledge Distillation in Vision Transformers: A Critical Reviewcited by 13
  2. 2022TRT-ViT: TensorRT-oriented Vision Transformercited by 10
  3. 2023NVAutoNet: Fast and Accurate 360circ 3D Visual Perception For Self Drivingcited by 6
  4. 2024Multi-Objective Neural Architecture Search by Learning Search Space Partitionscited by 1
  5. 2024AdaScale: Dynamic Context-aware DNN Scaling via Automated Adaptation Loop on Mobile Devicescited by 1
  6. 2022Tiered Pruning for Efficient Differentialble Inference-Aware Neural Architecture Searchcited by 0
  7. 2024Exploring Camera Encoder Designs for Autonomous Driving Perceptioncited by 0