mlsanityData to 7 Oct 2026

β-DARTS: Beta-Decay Regularization for Differentiable Architecture Search

arXiv 2203.01665 ↗First posted 3 Mar 2022cs.LG cs.AIcited by 63

Not on the map. Why?

Abstract (arXiv)

Neural Architecture Search~(NAS) has attracted increasingly more attention in recent years because of its capability to design deep neural networks automatically. Among them, differential NAS approaches such as DARTS, have gained popularity for the search efficiency. However, they suffer from two main issues, the weak robustness to the performance collapse and the poor generalization ability of the searched architectures. To solve these two problems, a simple-but-efficient regularization method, termed as Beta-Decay, is proposed to regularize the DARTS-based NAS searching process. Specifically, Beta-Decay regularization can impose constraints to keep the value and variance of activated architecture parameters from too large. Furthermore, we provide in-depth theoretical analysis on how it works and why it works. Experimental results on NAS-Bench-201 show that our proposed method can help to stabilize the searching process and makes the searched network more transferable across different datasets. In addition, our search scheme shows an outstanding property of being less dependent on training time and data. Comprehensive experiments on a variety of search spaces and datasets validate the effectiveness of the proposed method.

Key facts. Cited by 63 papers in the map (data through 7 Oct 2026). Builds on 25 of its 32 references that are in the map. First posted on arXiv 3 Mar 2022.

Builds on

25 of 32 references.

References · 25

  1. 2018DARTS: Differentiable Architecture Searchcited by 1,782
  2. 2020Sharpness-Aware Minimization for Efficiently Improving Generalizationcited by 1,404
  3. 2017Learning Transferable Architectures for Scalable Image Recognitioncited by 1,097
  4. 2018MnasNet: Platform-Aware Neural Architecture Search for Mobilecited by 937
  5. 2018Regularized Evolution for Image Classifier Architecture Searchcited by 902
  6. 2017Exploring Generalization in Deep Learningcited by 595
  7. 2020NAS-Bench-201: Extending the Scope of Reproducible Neural Architecture Searchcited by 412
  8. 2019Single Path One-Shot Neural Architecture Search with Uniform Samplingcited by 411
  9. 2018SNAS: Stochastic Neural Architecture Searchcited by 288
  10. 2019PC-DARTS: Partial Channel Connections for Memory-Efficient Architecture Searchcited by 265
  11. 2019Searching for A Robust Neural Architecture in Four GPU Hourscited by 258
  12. 2019Progressive Differentiable Architecture Search: Bridging the Depth Gap between Search and Evaluationcited by 230
13 more
  1. 2019Understanding and Robustifying Differentiable Architecture Searchcited by 176
  2. 2019DARTS+: Improved Differentiable Architecture Search with Early Stoppingcited by 105
  3. 2020Stabilizing Differentiable Architecture Search via Perturbation-based Regularizationcited by 102
  4. 2021Rethinking Architecture Selection in Differentiable NAScited by 92
  5. 2020DARTS-: Robustly Stepping out of Performance Collapse Without Indicatorscited by 81
  6. 2020DSNAS: Direct Neural Architecture Search without Parameter Retrainingcited by 69
  7. 2020NAS-Bench-1Shot1: Benchmarking and Dissecting One-shot Neural Architecture Searchcited by 64
  8. 2021iDARTS: Differentiable Architecture Search with Stochastic Implicit Gradientscited by 51
  9. 2018Path-Level Network Transformation for Efficient Architecture Searchcited by 43
  10. 2020Theory-Inspired Path-Regularized Differential Network Architecture Searchcited by 25
  11. 2020DOTS: Decoupling Operation and Topology in Differentiable Architecture Searchcited by 23
  12. 2022DropNAS: Grouped Operation Dropout for Differentiable Architecture Searchcited by 12
  13. 2021MixSearch: Searching for Domain Generalized Medical Image Segmentation Architecturescited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 8; 2023: 15; 2024: 20; 2025: 16; 2026: 4.

63 papers.

Sort:

By area:

  1. 2024EMR-Merging: Tuning-Free High-Performance Model Mergingcited by 108
  2. 2023Can GPT-4 Perform Neural Architecture Search?cited by 74
  3. 2024ClipSAM: CLIP and SAM Collaboration for Zero-Shot Anomaly Segmentationcited by 49
  4. 2025DiEP: Adaptive Mixture-of-Experts Compression through Differentiable Expert Pruningcited by 24
  5. 2022Lambda-DARTS: Mitigating Performance Collapse by Harmonizing Operation Selection among Cellscited by 15
  6. 2022Generalization Properties of NAS under Activation and Skip Connection Searchcited by 14
  7. 2024Multi-conditioned Graph Diffusion for Neural Architecture Searchcited by 13
  8. 2024LeMo-NADe: Multi-Parameter Neural Architecture Discovery with LLMscited by 11
  9. 2022Stimulative Training of Residual Networks: A Social Psychology Perspective of Loafingcited by 10
  10. 2023Performance-aware Approximation of Global Channel Pruning for Multitask CNNscited by 9
  11. 2024Once for Both: Single Stage of Importance and Sparsity Search for Vision Transformer Compressioncited by 8
  12. 2024NADER: Neural Architecture Design via Multi-Agent Collaborationcited by 8
38 more
  1. 2023β-DARTS++: Bi-level Regularization for Proxy-robust Differentiable Architecture Searchcited by 7
  2. 2025SEKI: Self-Evolution and Knowledge Inspiration based Neural Architecture Search via Large Language Modelscited by 7
  3. 2023Improving Differentiable Architecture Search via Self-Distillationcited by 6
  4. 2023Stimulative Training++: Go Beyond The Performance Limits of Residual Networkscited by 6
  5. 2024Robust NAS under adversarial training: benchmark, theory, and beyondcited by 6
  6. 2023Efficient Automation of Neural Network Design: A Survey on Differentiable Neural Architecture Searchcited by 5
  7. 2024Insights from the Use of Previously Unseen Neural Architecture Search Datasetscited by 5
  8. 2022Deep Architecture Connectivity Matters for Its Convergence: A Fine-Grained Analysiscited by 4
  9. 2024Robustifying and Boosting Training-Free Neural Architecture Searchcited by 4
  10. 2024OStr-DARTS: Differentiable Neural Architecture Search based on Operation Strengthcited by 4
  11. 2025Subnet-Aware Dynamic Supernet Training for Neural Architecture Searchcited by 4
  12. 2023DCLP: Neural Architecture Predictor with Curriculum Contrastive Learningcited by 3
  13. 2023Boosting Residual Networks with Group Knowledgecited by 3
  14. 2022MRF-UNets: Searching UNet with Markov Random Fieldscited by 2
  15. 2022DASS: Differentiable Architecture Search for Sparse neural networkscited by 2
  16. 2023A2S-NAS: Asymmetric Spectral-Spatial Neural Architecture Search For Hyperspectral Image Classificationcited by 2
  17. 2023Rethink DARTS Search Space and Renovate a New Benchmarkcited by 2
  18. 2023Efficient Architecture Search via Bi-level Data Pruningcited by 2
  19. 2024Enhanced Sparsification via Stimulative Trainingcited by 2
  20. 2024A Pairwise Comparison Relation-assisted Multi-objective Evolutionary Neural Architecture Search Method with Multi-population Mechanismcited by 2
  21. 2025Architecture-Aware Minimization (A2M): How to Find Flat Minima in Neural Architecture Searchcited by 2
  22. 2022α DARTS Once More: Enhancing Differentiable Architecture Search by Masked Image Modelingcited by 1
  23. 2022DAS: Neural Architecture Search via Distinguishing Activation Scorecited by 1
  24. 2023NASiam: Efficient Representation Learning using Neural Architecture Search for Siamese Networkscited by 1
  25. 2023Small Temperature is All You Need for Differentiable Architecture Searchcited by 1
  26. 2024Implantable Adaptive Cells: A Novel Enhancement for Pre-Trained U-Nets in Medical Image Segmentationcited by 1
  27. 2024TopoNAS: Boosting Search Efficiency of Gradient-based NAS via Topological Simplificationcited by 1
  28. 2024Revisiting Long-Tailed Learning: Insights from an Architectural Perspectivecited by 1
  29. 2025Regularizing Differentiable Architecture Search with Smooth Activationcited by 1
  30. 2025Dextr: Zero-Shot Neural Architecture Search with Singular Value Decomposition and Extrinsic Curvaturecited by 1
  31. 2025RevoNAD: Reflective Evolutionary Exploration for Neural Architecture Designcited by 1
  32. 2023JNDMix: JND-Based Data Augmentation for No-reference Image Quality Assessmentcited by 0
  33. 2023MGAS: Multi-Granularity Architecture Search for Trade-Off Between Model Effectiveness and Efficiencycited by 0
  34. 2024Causal-aware Graph Neural Architecture Search under Distribution Shiftscited by 0
  35. 2024The devil is in discretization discrepancy. Robustifying Differentiable NAS with Single-Stage Searching Protocolcited by 0
  36. 2024Pushing Joint Image Denoising and Classification to the Edgecited by 0
  37. 2024S2HPruner: Soft-to-Hard Distillation Bridges the Discretization Gap in Pruningcited by 0
  38. 2024SASE: A Searching Architecture for Squeeze and Excitation Operationscited by 0

50 most cited of 63 All 63 (JSON)