# iDARTS: Improving DARTS by Node Normalization and Decorrelation Discretization arXiv: https://arxiv.org/abs/2108.11014 ยท HTML: https://mlsanity.com/paper/2108.11014/ ## Key facts - Cited by 10 papers in the map (data through 14 Oct 2026). - Builds on 39 of its 47 references that are in the map. - First posted on arXiv 25 Aug 2021. ## Builds on (most cited first, 30 of 39) - [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016) - [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019) - [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018) - [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014) - [DARTS: Differentiable Architecture Search](https://mlsanity.com/paper/1806.09055/) (2018) - [Neural Architecture Search with Reinforcement Learning](https://mlsanity.com/paper/1611.01578/) (2016) - [ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design](https://mlsanity.com/paper/1807.11164/) (2018) - [Learning Transferable Architectures for Scalable Image Recognition](https://mlsanity.com/paper/1707.07012/) (2017) - [HyperNetworks](https://mlsanity.com/paper/1609.09106/) (2016) - [MnasNet: Platform-Aware Neural Architecture Search for Mobile](https://mlsanity.com/paper/1807.11626/) (2018) - [Regularized Evolution for Image Classifier Architecture Search](https://mlsanity.com/paper/1802.01548/) (2018) - [Efficient Neural Architecture Search via Parameter Sharing](https://mlsanity.com/paper/1802.03268/) (2018) - [ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware](https://mlsanity.com/paper/1812.00332/) (2018) - [Progressive Neural Architecture Search](https://mlsanity.com/paper/1712.00559/) (2017) - [Large-Scale Evolution of Image Classifiers](https://mlsanity.com/paper/1703.01041/) (2017) - [SNAS: Stochastic Neural Architecture Search](https://mlsanity.com/paper/1812.09926/) (2018) - [Designing Neural Network Architectures using Reinforcement Learning](https://mlsanity.com/paper/1611.02167/) (2016) - [Searching for A Robust Neural Architecture in Four GPU Hours](https://mlsanity.com/paper/1910.04465/) (2019) - [Progressive Differentiable Architecture Search: Bridging the Depth Gap between Search and Evaluation](https://mlsanity.com/paper/1904.12760/) (2019) - [SMASH: One-Shot Model Architecture Search through HyperNetworks](https://mlsanity.com/paper/1708.05344/) (2017) - [Neural Architecture Search with Bayesian Optimisation and Optimal Transport](https://mlsanity.com/paper/1802.07191/) (2018) - [Hierarchical Representations for Efficient Architecture Search](https://mlsanity.com/paper/1711.00436/) (2017) - [Understanding and Robustifying Differentiable Architecture Search](https://mlsanity.com/paper/1909.09656/) (2019) - [Neural Architecture Optimization](https://mlsanity.com/paper/1808.07233/) (2018) - [Genetic CNN](https://mlsanity.com/paper/1703.01513/) (2017) - [DARTS+: Improved Differentiable Architecture Search with Early Stopping](https://mlsanity.com/paper/1909.06035/) (2019) - [Stabilizing Differentiable Architecture Search via Perturbation-based Regularization](https://mlsanity.com/paper/2002.05283/) (2020) - [Rethinking Architecture Selection in Differentiable NAS](https://mlsanity.com/paper/2108.04392/) (2021) - [DSNAS: Direct Neural Architecture Search without Parameter Retraining](https://mlsanity.com/paper/2002.09128/) (2020) - [BayesNAS: A Bayesian Approach for Neural Architecture Search](https://mlsanity.com/paper/1905.04919/) (2019) ## Built on by (most cited first, 10 of 10) - [A Survey on Computationally Efficient Neural Architecture Search](https://mlsanity.com/paper/2206.01520/) (2022) - [SEKI: Self-Evolution and Knowledge Inspiration based Neural Architecture Search via Large Language Models](https://mlsanity.com/paper/2502.20422/) (2025) - [IS-DARTS: Stabilizing DARTS through Precise Measurement on Candidate Importance](https://mlsanity.com/paper/2312.12648/) (2023) - [OStr-DARTS: Differentiable Neural Architecture Search based on Operation Strength](https://mlsanity.com/paper/2409.14433/) (2024) - [A Pairwise Comparison Relation-assisted Multi-objective Evolutionary Neural Architecture Search Method with Multi-population Mechanism](https://mlsanity.com/paper/2407.15600/) (2024) - [SONATA: Self-adaptive Evolutionary Framework for Hardware-aware Neural Architecture Search](https://mlsanity.com/paper/2402.13204/) (2024) - [Exploring Complicated Search Spaces with Interleaving-Free Sampling](https://mlsanity.com/paper/2112.02488/) (2021) - [An Efficient End-to-End 3D Voxel Reconstruction based on Neural Architecture Search](https://mlsanity.com/paper/2202.13313/) (2022) - [Efficient Re-parameterization Operations Search for Easy-to-Deploy Network Based on Directional Evolutionary Strategy](https://mlsanity.com/paper/2204.06403/) (2022) - [Neural Architecture Search with Mixed Bio-inspired Learning Rules](https://mlsanity.com/paper/2507.13485/) (2025) ## Abstract (arXiv) Differentiable ARchiTecture Search (DARTS) uses a continuous relaxation of network representation and dramatically accelerates Neural Architecture Search (NAS) by almost thousands of times in GPU-day. However, the searching process of DARTS is unstable, which suffers severe degradation when training epochs become large, thus limiting its application. In this paper, we claim that this degradation issue is caused by the imbalanced norms between different nodes and the highly correlated outputs from various operations. We then propose an improved version of DARTS, namely iDARTS, to deal with the two problems. In the training phase, it introduces node normalization to maintain the norm balance. In the discretization phase, the continuous architecture is approximated based on the similarity between the outputs of the node and the decorrelated operations rather than the values of the architecture parameters. Extensive evaluation is conducted on CIFAR-10 and ImageNet, and the error rates of 2.25% and 24.7% are reported within 0.2 and 1.9 GPU-day for architecture search respectively, which shows its effectiveness. Additional analysis also reveals that iDARTS has the advantage in robustness and generalization over other DARTS-based counterparts.