mlsanityData to 14 Oct 2026

LP-NAS: Linear Programming-based Neural Architecture Search

arXiv 2608.14472 ↗First posted 14 Aug 2026cs.LG cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Neural Architecture Search (NAS) aims to automate neural network architecture design, reducing reliance on human expertise. Among the various NAS methods, differentiable NAS has gained prominence due to its efficiency and accuracy compared to conventional NAS approaches. Since differentiable NAS relaxes the architecture search space into a continuous domain, it is possible to apply principles from continuous optimization to NAS. In this paper, we propose Linear Programming-based NAS (LP-NAS), a mathematical programming-based framework for differentiable NAS that is applicable to a wide range of continuous search spaces. LP-NAS formulates a linear program (LP) using the validation-loss gradient and the training-loss Hessian to compute an architecture update direction that improves generalization while preserving the optimality of the model parameters. By following this LP-derived descent direction, LP-NAS efficiently navigates the architecture search space, leading to faster and more effective architecture optimization. We introduce two computationally efficient variants of LP-NAS, namely S-LP-NAS and R-LP-NAS. Applying LP-NAS to the Differentiable Architecture Search (DARTS) search space results in two algorithmic variants, S-LP-DARTS and R-LP-DARTS. Both variants achieve faster convergence and significantly higher validation performance during the early search iterations than the standard DARTS algorithm. Extensive experiments on CIFAR-10 and CIFAR-100 show that LP-DARTS outperforms standard DARTS in both the architecture search and evaluation phases. Additionally, we compare our approach with several DARTS variants (P-DARTS, PC-DARTS, and STO-DARTS) on the CIFAR-10 dataset and demonstrate its effectiveness. Furthermore, we validate the transferability of the discovered architectures through experiments on the ImageNet dataset.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 33 of its 47 references that are in the map. First posted on arXiv 14 Aug 2026.

Builds on

33 of 47 references.

References · 33

  1. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
  2. 2018DARTS: Differentiable Architecture Searchcited by 1,798
  3. 2016Neural Architecture Search with Reinforcement Learningcited by 1,499
  4. 2017Learning Transferable Architectures for Scalable Image Recognitioncited by 1,101
  5. 2018MnasNet: Platform-Aware Neural Architecture Search for Mobilecited by 943
  6. 2018Regularized Evolution for Image Classifier Architecture Searchcited by 909
  7. 2018Efficient Neural Architecture Search via Parameter Sharingcited by 837
  8. 2019Once-for-All: Train One Network and Specialize it for Efficient Deploymentcited by 787
  9. 2018ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardwarecited by 673
  10. 2018FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Searchcited by 485
  11. 2018Bilevel Programming for Hyperparameter Optimization and Meta-Learningcited by 405
  12. 2019PC-DARTS: Partial Channel Connections for Memory-Efficient Architecture Searchcited by 265
21 more
  1. 2019Auto-DeepLab: Hierarchical Neural Architecture Search for Semantic Image Segmentationcited by 260
  2. 2020A Comprehensive Survey of Neural Architecture Search: Challenges and Solutionscited by 254
  3. 2019Progressive Differentiable Architecture Search: Bridging the Depth Gap between Search and Evaluationcited by 230
  4. 2021AutoFormer: Searching Transformers for Visual Recognitioncited by 188
  5. 2019Understanding and Robustifying Differentiable Architecture Searchcited by 178
  6. 2020A Survey on Evolutionary Neural Architecture Searchcited by 142
  7. 2023Neural Architecture Search: Insights from 1000 Paperscited by 139
  8. 2019Fair DARTS: Eliminating Unfair Advantages in Differentiable Architecture Searchcited by 131
  9. 2017A Review on Bilevel Optimization: From Classical to Evolutionary Approaches and Applicationscited by 129
  10. 2019DARTS+: Improved Differentiable Architecture Search with Early Stoppingcited by 105
  11. 2019GraphNAS: Graph Neural Architecture Search with Reinforcement Learningcited by 41
  12. 2021Inexact bilevel stochastic gradient methods for constrained and unconstrained lower-level problemscited by 11
  13. 2024LLaMA-NAS: Efficient Neural Architecture Search for Large Language Modelscited by 9
  14. 2021Enhanced Gradient for Differentiable Architecture Searchcited by 5
  15. 2024Relax DARTS: Relaxing the Constraints of Differentiable Architecture Search for Eye Movement Recognitioncited by 4
  16. 2024OStr-DARTS: Differentiable Neural Architecture Search based on Operation Strengthcited by 4
  17. 2024Evolution and Efficiency in Neural Architecture Search: Bridging the Gap Between Expert Design and Automated Optimizationcited by 3
  18. 2026LiFT: Local Search via Linear Programming for Overfitting-Controlled Transformerscited by 3
  19. 2024A Lightweight Neural Architecture Search Model for Medical Image Classificationcited by 2
  20. 2025A Review of Bilevel Optimization: Methods, Emerging Applications, and Recent Advancementscited by 2
  21. 2026Bilevel Optimization for Neural Architecture Searchcited by 2

Built on by

Not cited yet.