mlsanityData to 7 Oct 2026

MiLeNAS: Efficient Neural Architecture Search via Mixed-Level Reformulation

arXiv 2003.12238 ↗First posted 27 Mar 2020cs.LG cs.CV stat.MLcited by 56

Not on the map. Why?

Abstract (arXiv)

Many recently proposed methods for Neural Architecture Search (NAS) can be formulated as bilevel optimization. For efficient implementation, its solution requires approximations of second-order methods. In this paper, we demonstrate that gradient errors caused by such approximations lead to suboptimality, in the sense that the optimization procedure fails to converge to a (locally) optimal solution. To remedy this, this paper proposes mldas, a mixed-level reformulation for NAS that can be optimized efficiently and reliably. It is shown that even when using a simple first-order method on the mixed-level formulation, mldas can achieve a lower validation error for NAS problems. Consequently, architectures obtained by our method achieve consistently higher accuracies than those obtained from bilevel optimization. Moreover, mldas proposes a framework beyond DARTS. It is upgraded via model size-based search and early stopping strategies to complete the search process in around 5 hours. Extensive experiments within the convolutional architecture search space validate the effectiveness of our approach.

Key facts. Cited by 56 papers in the map (data through 7 Oct 2026). First posted on arXiv 27 Mar 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 25; 2022: 11; 2023: 9; 2024: 3; 2025: 6; 2026: 2.

56 papers.

Sort:

By area:

  1. 2021Investigating Bi-Level Optimization for Learning and Vision from a Unified Perspective: A Survey and Beyondcited by 151
  2. 2021Parameter Prediction for Unseen Deep Architecturescited by 89
  3. 2021FedNLP: Benchmarking Federated Learning Methods for Natural Language Processing Taskscited by 84
  4. 2021From Distributed Machine Learning to Federated Learning: A Surveycited by 65
  5. 2021Lower Bounds and Accelerated Algorithms for Bilevel Optimizationcited by 46
  6. 2021PocketNet: Extreme Lightweight Face Recognition Network using Neural Architecture Search and Multi-Step Knowledge Distillationcited by 36
  7. 2023Visual Tuningcited by 34
  8. 2021KNAS: Green Neural Architecture Searchcited by 33
  9. 2021FedCV: A Federated Learning Framework for Diverse Computer Vision Taskscited by 32
  10. 2021Towards Improving the Consistency, Efficiency, and Flexibility of Differentiable Neural Architecture Searchcited by 27
  11. 2021Federated Learning for Internet of Things: A Federated Learning Framework for On-device Anomaly Data Detectioncited by 19
  12. 2021Linear, or Non-Linear, That is the Question!cited by 19
38 more
  1. 2025PreGenie: An Agentic Framework for High-quality Visual Presentation Generationcited by 14
  2. 2022DropNAS: Grouped Operation Dropout for Differentiable Architecture Searchcited by 12
  3. 2021Self-supervised Cross-silo Federated Neural Architecture Searchcited by 11
  4. 2021FedMe: Federated Learning via Model Exchangecited by 10
  5. 2021Reconsidering CO2 emissions from Computer Visioncited by 9
  6. 2021Fisher Task Distance and Its Application in Neural Architecture Searchcited by 7
  7. 2025SEKI: Self-Evolution and Knowledge Inspiration based Neural Architecture Search via Large Language Modelscited by 7
  8. 2021Learning Calibrated-Guidance for Object Detection in Aerial Imagescited by 6
  9. 2021L2NAS: Learning to Optimize Neural Architectures via Continuous-Action Reinforcement Learningcited by 6
  10. 2022Automatic Relation-aware Graph Network Proliferationcited by 6
  11. 2023TransNAS-TSAD: Harnessing Transformers for Multi-Objective Neural Architecture Search in Time Series Anomaly Detectioncited by 6
  12. 2024Differentiable Combinatorial Scheduling at Scalecited by 6
  13. 2021Neural Architecture Search via Combinatorial Multi-Armed Banditcited by 5
  14. 2021Contrastive Self-supervised Neural Architecture Searchcited by 5
  15. 2021FedGraphNN: A Federated Learning System and Benchmark for Graph Neural Networkscited by 5
  16. 2021Differentiable Neural Architecture Search for Extremely Lightweight Image Super-Resolutioncited by 5
  17. 2022FocusFormer: Focusing on What We Need via Architecture Samplercited by 5
  18. 2022Physics-aware Differentiable Discrete Codesign for Diffractive Optical Neural Networkscited by 5
  19. 2022Pareto-aware Neural Architecture Generation for Diverse Computational Budgetscited by 4
  20. 2021A3C-S: Automated Agent Accelerator Co-Search towards Efficient Deep Reinforcement Learningcited by 3
  21. 2022Towards Tailored Models on Private AIoT Devices: Federated Direct Neural Architecture Searchcited by 3
  22. 2022FEATHERS: Federated Architecture and Hyperparameter Searchcited by 3
  23. 2022Multi-Agent Automated Machine Learningcited by 3
  24. 2023Divide-and-Conquer the NAS puzzle in Resource Constrained Federated Learning Systemscited by 3
  25. 2023Search-time Efficient Device Constraints-Aware Neural Architecture Searchcited by 3
  26. 2021CONet: Channel Optimization for Convolutional Neural Networkscited by 2
  27. 2021RT-RCG: Neural Network and Accelerator Search Towards Effective and Real-time ECG Reconstruction from Intracardiac Electrogramscited by 2
  28. 2022Cross Task Neural Architecture Search for EEG Signal Classificationscited by 2
  29. 2023Rethink DARTS Search Space and Renovate a New Benchmarkcited by 2
  30. 2024A Lightweight Neural Architecture Search Model for Medical Image Classificationcited by 2
  31. 2023DREAM: Debugging and Repairing AutoML Pipelinescited by 1
  32. 2025ZO-DARTS++: An Efficient and Size-Variable Zeroth-Order Neural Architecture Search Algorithmcited by 1
  33. 2021Is Differentiable Architecture Search truly a One-Shot Method?cited by 0
  34. 2022FAQS: Communication-efficient Federate DNN Architecture and Quantization Co-Search for personalized Hardware-aware Preferencescited by 0
  35. 2022CPMLHO:Hyperparameter Tuning via Cutting Plane and Mixed-Level Optimizationcited by 0
  36. 2023Automated Search-Space Generation Neural Architecture Searchcited by 0
  37. 2023A Survey on Multi-Objective Neural Architecture Searchcited by 0
  38. 2023AdaNAS: Adaptively Post-processing with Self-supervised Neural Architecture Search for Ensemble Rainfall Forecastscited by 0

50 most cited of 56 All 56 (JSON)