# FBNetV2: Differentiable Neural Architecture Search for Spatial and Channel Dimensions

arXiv: https://arxiv.org/abs/2004.05565 · HTML: https://mlsanity.com/paper/2004.05565/

## Key facts

- Cited by 152 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Apr 2020.

## Built on by (most cited first, 30 of 152)

- [HW-NAS-Bench:Hardware-Aware Neural Architecture Search Benchmark](https://mlsanity.com/paper/2103.10584/) (2021)
- [A Review of the Gumbel-max Trick and its Extensions for Discrete Stochasticity in Machine Learning](https://mlsanity.com/paper/2110.01515/) (2021)
- [Full Stack Optimization of Transformer Inference: a Survey](https://mlsanity.com/paper/2302.14017/) (2023)
- [Can GPT-4 Perform Neural Architecture Search?](https://mlsanity.com/paper/2304.10970/) (2023)
- [Zen-NAS: A Zero-Shot NAS for High-Performance Deep Image Recognition](https://mlsanity.com/paper/2102.01063/) (2021)
- [Model Compression and Efficient Inference for Large Language Models: A Survey](https://mlsanity.com/paper/2402.09748/) (2024)
- [BossNAS: Exploring Hybrid CNN-transformers with Block-wisely Self-supervised Neural Architecture Search](https://mlsanity.com/paper/2103.12424/) (2021)
- [ZiCo: Zero-shot NAS via Inverse Coefficient of Variation on Gradients](https://mlsanity.com/paper/2301.11300/) (2023)
- [Better Teacher Better Student: Dynamic Prior Knowledge for Knowledge Distillation](https://mlsanity.com/paper/2206.06067/) (2022)
- [Lightweight Deep Learning for Resource-Constrained Environments: A Survey](https://mlsanity.com/paper/2404.07236/) (2024)
- [Data Efficient Language-supervised Zero-shot Recognition with Optimal Transport Distillation](https://mlsanity.com/paper/2112.09445/) (2021)
- [Contrastive Neural Architecture Search with Neural Architecture Comparators](https://mlsanity.com/paper/2103.05471/) (2021)
- [Learned Smartphone ISP on Mobile NPUs with Deep Learning, Mobile AI 2021 Challenge: Report](https://mlsanity.com/paper/2105.07809/) (2021)
- [Real-Time Quantized Image Super-Resolution on Mobile NPUs, Mobile AI 2021 Challenge: Report](https://mlsanity.com/paper/2105.07825/) (2021)
- [ViTAS: Vision Transformer Architecture Search](https://mlsanity.com/paper/2106.13700/) (2021)
- [Advancing Sequential Numerical Prediction in Autoregressive Models](https://mlsanity.com/paper/2505.13077/) (2025)
- [DiEP: Adaptive Mixture-of-Experts Compression through Differentiable Expert Pruning](https://mlsanity.com/paper/2509.16105/) (2025)
- [K-shot NAS: Learnable Weight-Sharing for NAS with K-shot Supernets](https://mlsanity.com/paper/2106.06442/) (2021)
- [PSViT: Better Vision Transformer via Token Pooling and Attention Sharing](https://mlsanity.com/paper/2108.03428/) (2021)
- [BN-NAS: Neural Architecture Search with Batch Normalization](https://mlsanity.com/paper/2108.07375/) (2021)
- [Zero-Shot Neural Architecture Search: Challenges, Solutions, and Opportunities](https://mlsanity.com/paper/2307.01998/) (2023)
- [Compiler-Aware Neural Architecture Search for On-Mobile Real-time Super-Resolution](https://mlsanity.com/paper/2207.12577/) (2022)
- [Fast and Accurate Single-Image Depth Estimation on Mobile Devices, Mobile AI 2021 Challenge: Report](https://mlsanity.com/paper/2105.08630/) (2021)
- [Real-Time Video Super-Resolution on Smartphones with Deep Learning, Mobile AI 2021 Challenge: Report](https://mlsanity.com/paper/2105.08826/) (2021)
- [When, where, and how to add new neurons to ANNs](https://mlsanity.com/paper/2202.08539/) (2022)
- [Rethinking Centered Kernel Alignment in Knowledge Distillation](https://mlsanity.com/paper/2401.11824/) (2024)
- [HELP: Hardware-Adaptive Efficient Latency Prediction for NAS via Meta-Learning](https://mlsanity.com/paper/2106.08630/) (2021)
- [Efficient and Accurate Quantized Image Super-Resolution on Mobile NPUs, Mobile AI & AIM 2022 challenge: Report](https://mlsanity.com/paper/2211.05910/) (2022)
- [Efficient Modulation for Vision Networks](https://mlsanity.com/paper/2403.19963/) (2024)
- [Searching by Generating: Flexible and Efficient One-Shot NAS with Architecture Generator](https://mlsanity.com/paper/2103.07289/) (2021)

## Abstract (arXiv)

Differentiable Neural Architecture Search (DNAS) has demonstrated great success in designing state-of-the-art, efficient neural networks. However, DARTS-based DNAS's search space is small when compared to other search methods', since all candidate network layers must be explicitly instantiated in memory. To address this bottleneck, we propose a memory and computationally efficient DNAS variant: DMaskingNAS. This algorithm expands the search space by up to 1014× over conventional DNAS, supporting searches over spatial and channel dimensions that are otherwise prohibitively expensive: input resolution and number of filters. We propose a masking mechanism for feature map reuse, so that memory and computational costs stay nearly constant as the search space expands. Furthermore, we employ effective shape propagation to maximize per-FLOP or per-parameter accuracy. The searched FBNetV2s yield state-of-the-art performance when compared with all previous architectures. With up to 421× less search cost, DMaskingNAS finds models with 0.9% higher accuracy, 15% fewer FLOPs than MobileNetV3-Small; and with similar accuracy but 20% fewer FLOPs than Efficient-B0. Furthermore, our FBNetV2 outperforms MobileNetV3 by 2.6% in accuracy, with equivalent model size. FBNetV2 models are open-sourced at https://github.com/facebookresearch/mobile-vision.