mlsanityData to 7 Oct 2026

Can weight sharing outperform random architecture search? An investigation with TuNAS

arXiv 2008.06120 ↗First posted 13 Aug 2020cs.LG cs.CV stat.MLcited by 68

Not on the map. Why?

Abstract (arXiv)

Efficient Neural Architecture Search methods based on weight sharing have shown good promise in democratizing Neural Architecture Search for computer vision models. There is, however, an ongoing debate whether these efficient methods are significantly better than random search. Here we perform a thorough comparison between efficient and random search methods on a family of progressively larger and more challenging search spaces for image classification and detection on ImageNet and COCO. While the efficacies of both methods are problem-dependent, our experiments demonstrate that there are large, realistic tasks where efficient search methods can provide substantial gains over random search. In addition, we propose and evaluate techniques which improve the quality of searched architectures and reduce the need for manual hyper-parameter tuning. Source code and experiment data are available at https://github.com/google-research/google-research/tree/master/tunas

Key facts. Cited by 68 papers in the map (data through 7 Oct 2026). First posted on arXiv 13 Aug 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 27; 2022: 21; 2023: 11; 2024: 5; 2025: 3; 2026: 1.

68 papers.

Sort:

By area:

  1. 2021Primer: Searching for Efficient Transformers for Language Modelingcited by 162
  2. 2023Neural Architecture Search: Insights from 1000 Paperscited by 136
  3. 2021MoViNets: Mobile Video Networks for Efficient Video Recognitioncited by 135
  4. 2023EvoPrompting: Language Models for Code-Level Neural Architecture Searchcited by 129
  5. 2024MobileNetV4 -- Universal Models for the Mobile Ecosystemcited by 98
  6. 2021BossNAS: Exploring Hybrid CNN-transformers with Block-wisely Self-supervised Neural Architecture Searchcited by 63
  7. 2021A Comprehensive Survey on Hardware-Aware Neural Architecture Searchcited by 60
  8. 2022Packed-Ensembles for Efficient Uncertainty Estimationcited by 49
  9. 2022On the Factory Floor: ML Engineering for Industrial-Scale Ads Recommendation Modelscited by 34
  10. 2022A Survey on Video Action Recognition in Sports: Datasets, Methods and Applicationscited by 27
  11. 2021Searching for Fast Model Families on Datacenter Acceleratorscited by 22
  12. 2021CATE: Computation-aware Neural Architecture Encoding with Transformerscited by 21
38 more
  1. 2021PyGlove: Symbolic Programming for Automated Machine Learningcited by 19
  2. 2021Pi-NAS: Improving Neural Architecture Search by Reducing Supernet Training Consistency Shiftcited by 14
  3. 2021HyperSPNs: Compact and Expressive Probabilistic Circuitscited by 14
  4. 2023Unified Embedding: Battle-Tested Feature Representations for Web-Scale ML Systemscited by 14
  5. 2021NetAdaptV2: Efficient Neural Architecture Search with Fast Super-Network Training and Architecture Optimizationcited by 13
  6. 2021Automated Deep Learning: Neural Architecture Search Is Not the Endcited by 13
  7. 2021Rethinking Co-design of Neural Architectures and Hardware Acceleratorscited by 12
  8. 2022Neural Architecture Search for Dense Prediction Tasks in Computer Visioncited by 11
  9. 2024HW-GPT-Bench: Hardware-Aware Architecture Benchmark for Language Modelscited by 10
  10. 2021Generative Adversarial Neural Architecture Searchcited by 9
  11. 2021Searching for Efficient Multi-Stage Vision Transformerscited by 9
  12. 2021Landmark Regularization: Ranking Guided Super-Net Training in Neural Architecture Searchcited by 8
  13. 2021One Proxy Device Is Enough for Hardware-Aware Neural Architecture Searchcited by 8
  14. 2021Fisher Task Distance and Its Application in Neural Architecture Searchcited by 7
  15. 2022NASRec: Weight Sharing Neural Architecture Search for Recommender Systemscited by 7
  16. 2023Enhancing Neural Architecture Search with Multiple Hardware Constraints for Deep Learning Model Deployment on Tiny IoT Devicescited by 7
  17. 2021L2NAS: Learning to Optimize Neural Architectures via Continuous-Action Reinforcement Learningcited by 6
  18. 2022TabNAS: Rejection Sampling for Neural Architecture Search on Tabular Datasetscited by 6
  19. 2024Increasing Model Capacity for Free: A Simple Strategy for Parameter Efficient Fine-tuningcited by 6
  20. 2022A Neural Phillips Curve and a Deep Output Gapcited by 5
  21. 2022Neural Architecture Search for Energy Efficient Always-on Audio Modelscited by 5
  22. 2022FocusFormer: Focusing on What We Need via Architecture Samplercited by 5
  23. 2022GraphPNAS: Learning Distribution of Good Neural Architectures via Deep Graph Generative Modelscited by 5
  24. 2023FLIQS: One-Shot Mixed-Precision Floating-Point and Integer Quantization Searchcited by 5
  25. 2024einspace: Searching for Neural Architectures from Fundamental Operationscited by 5
  26. 2021Bag of Tricks for Neural Architecture Searchcited by 4
  27. 2021Profiling Neural Blocks and Design Spaces for Mobile Neural Architecture Searchcited by 4
  28. 2022GradTail: Learning Long-Tailed Data Using Gradient-based Sample Weightingcited by 4
  29. 2022Searching for Efficient Neural Architectures for On-Device ML on Edge TPUscited by 4
  30. 2022Neural Architecture Search using Property Guided Synthesiscited by 4
  31. 2022LidarNAS: Unifying and Searching Neural Architectures for 3D Point Cloudscited by 4
  32. 2022Pareto-aware Neural Architecture Generation for Diverse Computational Budgetscited by 4
  33. 2022Towards Hardware-Specific Automatic Compression of Neural Networkscited by 4
  34. 2023Transfer-Once-For-All: AI Model Optimization for Edgecited by 4
  35. 2023MaskConver: Revisiting Pure Convolution Model for Panoptic Segmentationcited by 4
  36. 2021Pareto-Frontier-aware Neural Architecture Generation for Diverse Budgetscited by 3
  37. 2022When NAS Meets Trees: An Efficient Algorithm for Neural Architecture Searchcited by 3
  38. 2023LayerNAS: Neural Architecture Search in Polynomial Complexitycited by 3

50 most cited of 68 All 68 (JSON)