# Searching for Fast Model Families on Datacenter Accelerators

arXiv: https://arxiv.org/abs/2102.05610 · HTML: https://mlsanity.com/paper/2102.05610/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 45 of its 67 references that are in the map.
- First posted on arXiv 10 Feb 2021.

## Builds on (most cited first, 30 of 45)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [Wide Residual Networks](https://mlsanity.com/paper/1605.07146/) (2016)
- [Learning both Weights and Connections for Efficient Neural Networks](https://mlsanity.com/paper/1506.02626/) (2015)
- [Searching for MobileNetV3](https://mlsanity.com/paper/1905.02244/) (2019)
- [DARTS: Differentiable Architecture Search](https://mlsanity.com/paper/1806.09055/) (2018)
- [Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning](https://mlsanity.com/paper/1602.07261/) (2016)
- [Neural Architecture Search with Reinforcement Learning](https://mlsanity.com/paper/1611.01578/) (2016)
- [Learning Transferable Architectures for Scalable Image Recognition](https://mlsanity.com/paper/1707.07012/) (2017)
- [SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size](https://mlsanity.com/paper/1602.07360/) (2016)
- [Designing Network Design Spaces](https://mlsanity.com/paper/2003.13678/) (2020)
- [MnasNet: Platform-Aware Neural Architecture Search for Mobile](https://mlsanity.com/paper/1807.11626/) (2018)
- [Regularized Evolution for Image Classifier Architecture Search](https://mlsanity.com/paper/1802.01548/) (2018)
- [Efficient Neural Architecture Search via Parameter Sharing](https://mlsanity.com/paper/1802.03268/) (2018)
- [Once-for-All: Train One Network and Specialize it for Efficient Deployment](https://mlsanity.com/paper/1908.09791/) (2019)
- [ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware](https://mlsanity.com/paper/1812.00332/) (2018)
- [In-Datacenter Performance Analysis of a Tensor Processing Unit](https://mlsanity.com/paper/1704.04760/) (2017)
- [Learning Structured Sparsity in Deep Neural Networks](https://mlsanity.com/paper/1608.03665/) (2016)
- [Progressive Neural Architecture Search](https://mlsanity.com/paper/1712.00559/) (2017)
- [FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Search](https://mlsanity.com/paper/1812.03443/) (2018)
- [Single Path One-Shot Neural Architecture Search with Uniform Sampling](https://mlsanity.com/paper/1904.00420/) (2019)
- [AMC: AutoML for Model Compression and Acceleration on Mobile Devices](https://mlsanity.com/paper/1802.03494/) (2018)
- [Designing Neural Network Architectures using Reinforcement Learning](https://mlsanity.com/paper/1611.02167/) (2016)
- [Searching for A Robust Neural Architecture in Four GPU Hours](https://mlsanity.com/paper/1910.04465/) (2019)

## Built on by (most cited first, 22 of 22)

- [EfficientNetV2: Smaller Models and Faster Training](https://mlsanity.com/paper/2104.00298/) (2021)
- [Carbon Emissions and Large Neural Network Training](https://mlsanity.com/paper/2104.10350/) (2021)
- [TPU v4: An Optically Reconfigurable Supercomputer for Machine Learning with Hardware Support for Embeddings](https://mlsanity.com/paper/2304.01433/) (2023)
- [The Carbon Footprint of Machine Learning Training Will Plateau, Then Shrink](https://mlsanity.com/paper/2204.05149/) (2022)
- [Lightweight Deep Learning for Resource-Constrained Environments: A Survey](https://mlsanity.com/paper/2404.07236/) (2024)
- [A Full-Stack Search Technique for Domain Optimized Deep Learning Accelerators](https://mlsanity.com/paper/2105.12842/) (2021)
- [FBNetV5: Neural Architecture Search for Multiple Tasks in One Run](https://mlsanity.com/paper/2111.10007/) (2021)
- [Less is More: Selective Layer Finetuning with SubTuning](https://mlsanity.com/paper/2302.06354/) (2023)
- [Measuring what Really Matters: Optimizing Neural Networks for TinyML](https://mlsanity.com/paper/2104.10645/) (2021)
- [AutoDistill: an End-to-End Framework to Explore and Distill Hardware-Efficient Language Models](https://mlsanity.com/paper/2201.08539/) (2022)
- [GPUNet: Searching the Deployable Convolution Neural Networks for GPUs](https://mlsanity.com/paper/2205.00841/) (2022)
- [Composer: A Search Framework for Hybrid Neural Architecture Design](https://mlsanity.com/paper/2510.00379/) (2025)
- [Restructurable Activation Networks](https://mlsanity.com/paper/2208.08562/) (2022)
- [On Accelerating Edge AI: Optimizing Resource-Constrained Environments](https://mlsanity.com/paper/2501.15014/) (2025)
- [ScaleNet: Searching for the Model to Scale](https://mlsanity.com/paper/2207.07267/) (2022)
- [U-Boost NAS: Utilization-Boosted Differentiable Neural Architecture Search](https://mlsanity.com/paper/2203.12412/) (2022)
- [Revisiting Architecture-aware Knowledge Distillation: Smaller Models and Faster Search](https://mlsanity.com/paper/2206.13130/) (2022)
- [ISyNet: Convolutional Neural Networks design for AI accelerator](https://mlsanity.com/paper/2109.01932/) (2021)
- [Design Automation for Fast, Lightweight, and Effective Deep Learning Models: A Survey](https://mlsanity.com/paper/2208.10498/) (2022)
- [MSPT: A Lightweight Face Image Quality Assessment Method with Multi-stage Progressive Training](https://mlsanity.com/paper/2508.07590/) (2025)
- [Tiered Pruning for Efficient Differentialble Inference-Aware Neural Architecture Search](https://mlsanity.com/paper/2209.11785/) (2022)
- [Neural network scoring for efficient computing](https://mlsanity.com/paper/2310.09554/) (2023)

## Abstract (arXiv)

Neural Architecture Search (NAS), together with model scaling, has shown remarkable progress in designing high accuracy and fast convolutional architecture families. However, as neither NAS nor model scaling considers sufficient hardware architecture details, they do not take full advantage of the emerging datacenter (DC) accelerators. In this paper, we search for fast and accurate CNN model families for efficient inference on DC accelerators. We first analyze DC accelerators and find that existing CNNs suffer from insufficient operational intensity, parallelism, and execution efficiency. These insights let us create a DC-accelerator-optimized search space, with space-to-depth, space-to-batch, hybrid fused convolution structures with vanilla and depthwise convolutions, and block-wise activation functions. On top of our DC accelerator optimized neural architecture search space, we further propose a latency-aware compound scaling (LACS), the first multi-objective compound scaling method optimizing both accuracy and latency. Our LACS discovers that network depth should grow much faster than image size and network width, which is quite different from previous compound scaling results. With the new search space and LACS, our search and scaling on datacenter accelerators results in a new model series named EfficientNet-X. EfficientNet-X is up to more than 2X faster than EfficientNet (a model series with state-of-the-art trade-off on FLOPs and accuracy) on TPUv3 and GPUv100, with comparable accuracy. EfficientNet-X is also up to 7X faster than recent RegNet and ResNeSt on TPUv3 and GPUv100.