mlsanityData to 7 Oct 2026

Single Path One-Shot Neural Architecture Search with Uniform Sampling

arXiv 1904.00420 ↗First posted 31 Mar 2019cs.CVcited by 411

Not on the map. Why?

Abstract (arXiv)

We revisit the one-shot Neural Architecture Search (NAS) paradigm and analyze its advantages over existing NAS approaches. Existing one-shot method, however, is hard to train and not yet effective on large scale datasets like ImageNet. This work propose a Single Path One-Shot model to address the challenge in the training. Our central idea is to construct a simplified supernet, where all architectures are single paths so that weight co-adaption problem is alleviated. Training is performed by uniform path sampling. All architectures (and their weights) are trained fully and equally. Comprehensive experiments verify that our approach is flexible and effective. It is easy to train and fast to search. It effortlessly supports complex search spaces (e.g., building blocks, channel, mixed-precision quantization) and different search constraints (e.g., FLOPs, latency). It is thus convenient to use for various needs. It achieves start-of-the-art performance on the large dataset ImageNet.

Key facts. Cited by 411 papers in the map (data through 7 Oct 2026). First posted on arXiv 31 Mar 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 119; 2022: 99; 2023: 85; 2024: 57; 2025: 35; 2026: 16.

411 papers.

Sort:

By area:

  1. 2021Transformers in Vision: A Surveycited by 1,009
  2. 2021BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstructioncited by 464
  3. 2024LongRoPE: Extending LLM Context Window Beyond 2 Million Tokenscited by 302
  4. 2022Scaling & Shifting Your Features: A New Baseline for Efficient Model Tuningcited by 292
  5. 2024LocalMamba: Visual State Space Model with Windowed Selective Scancited by 235
  6. 2024MobileLLM: Optimizing Sub-billion Parameter Language Models for On-Device Use Casescited by 216
  7. 2021AutoFormer: Searching Transformers for Visual Recognitioncited by 186
  8. 2023EfficientViT: Memory Efficient Vision Transformer with Cascaded Group Attentioncited by 172
  9. 2021Neural Architecture Search on ImageNet in Four GPU Hours: A Theoretically Inspired Perspectivecited by 168
  10. 2022Scaling Up Your Kernels to 31x31: Revisiting Large Kernel Design in CNNscited by 155
  11. 2023Neural Architecture Search: Insights from 1000 Paperscited by 136
  12. 2023Structured Pruning for Deep Convolutional Neural Networks: A surveycited by 122
38 more
  1. 2021Involution: Inverting the Inherence of Convolution for Visual Recognitioncited by 99
  2. 2021QuantumNAS: Noise-Adaptive Search for Robust Quantum Circuitscited by 97
  3. 2021Rethinking Architecture Selection in Differentiable NAScited by 92
  4. 2021Sample and Computation Redistribution for Efficient Face Detectioncited by 86
  5. 2021Dynamic Slimmable Networkcited by 83
  6. 2021LightTrack: Finding Lightweight Neural Networks for Object Tracking via One-Shot Architecture Searchcited by 82
  7. 2023Full Stack Optimization of Transformer Inference: a Surveycited by 79
  8. 2023Can GPT-4 Perform Neural Architecture Search?cited by 74
  9. 2021Zen-NAS: A Zero-Shot NAS for High-Performance Deep Image Recognitioncited by 71
  10. 2023AutoDiffusion: Training-Free Optimization of Time Steps and Architectures for Automated Diffusion Model Accelerationcited by 70
  11. 2022Vision Transformer Slimming: Multi-Dimension Searching in Continuous Optimization Spacecited by 66
  12. 2021BossNAS: Exploring Hybrid CNN-transformers with Block-wisely Self-supervised Neural Architecture Searchcited by 63
  13. 2022β-DARTS: Beta-Decay Regularization for Differentiable Architecture Searchcited by 63
  14. 2023QLLM: Accurate and Efficient Low-Bitwidth Quantization for Large Language Modelscited by 63
  15. 2021A Comprehensive Survey on Hardware-Aware Neural Architecture Searchcited by 60
  16. 2023SparseViT: Revisiting Activation Sparsity for Efficient High-Resolution Vision Transformercited by 60
  17. 2023ZiCo: Zero-shot NAS via Inverse Coefficient of Variation on Gradientscited by 57
  18. 2021NAS-BERT: Task-Agnostic and Adaptive-Size BERT Compression with Neural Architecture Searchcited by 55
  19. 2021Anycost GANs for Interactive Image Synthesis and Editingcited by 54
  20. 2021GLiT: Neural Architecture Search for Global and Local Image Transformercited by 53
  21. 2022Mixed-Precision Neural Network Quantization via Learned Layer-wise Importancecited by 53
  22. 2022Neural Architecture Search for Spiking Neural Networkscited by 52
  23. 2022Divert More Attention to Vision-Language Trackingcited by 47
  24. 2023ZipLM: Inference-Aware Structured Pruning of Language Modelscited by 46
  25. 2022AutoSNN: Towards Energy-Efficient Spiking Neural Networkscited by 40
  26. 2024Tiny Machine Learning: Progress and Futurescited by 40
  27. 2021HR-NAS: Searching Efficient High-Resolution Neural Architectures with Lightweight Transformerscited by 39
  28. 2021Searching the Search Space of Vision Transformercited by 39
  29. 2021Automated Machine Learning on Graphs: A Surveycited by 38
  30. 2021Teachers Do More Than Teach: Compressing Image-to-Image Modelscited by 37
  31. 2021OMPQ: Orthogonal Mixed Precision Quantizationcited by 37
  32. 2021Pooling Architecture Search for Graph Classificationcited by 36
  33. 2022Q-ViT: Fully Differentiable Quantization for Vision Transformercited by 36
  34. 2022Automated Progressive Learning for Efficient Training of Vision Transformerscited by 36
  35. 2022SPDY: Accurate Pruning with Speedup Guaranteescited by 34
  36. 2021Neural Architecture Search with Random Labelscited by 33
  37. 2021Contrastive Neural Architecture Search with Neural Architecture Comparatorscited by 33
  38. 2022Training-free Transformer Architecture Searchcited by 33

50 most cited of 411 All 411 (JSON)