# Cyclic Differentiable Architecture Search

arXiv: https://arxiv.org/abs/2006.10724 · HTML: https://mlsanity.com/paper/2006.10724/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Jun 2020.

## Built on by (most cited first, 18 of 18)

- [InOR-Net: Incremental 3D Object Recognition Network for Point Cloud Representation](https://mlsanity.com/paper/2302.09886/) (2023)
- [Multi-Tailed Vision Transformer for Efficient Inference](https://mlsanity.com/paper/2203.01587/) (2022)
- [EEEA-Net: An Early Exit Evolutionary Neural Architecture Search](https://mlsanity.com/paper/2108.06156/) (2021)
- [β-DARTS++: Bi-level Regularization for Proxy-robust Differentiable Architecture Search](https://mlsanity.com/paper/2301.06393/) (2023)
- [Efficient Automation of Neural Network Design: A Survey on Differentiable Neural Architecture Search](https://mlsanity.com/paper/2304.05405/) (2023)
- [Differentiable Architecture Search with Random Features](https://mlsanity.com/paper/2208.08835/) (2022)
- [Single-DARTS: Towards Stable Architecture Search](https://mlsanity.com/paper/2108.08128/) (2021)
- [A Survey of Supernet Optimization and its Applications: Spatial and Temporal Optimization for Neural Architecture Search](https://mlsanity.com/paper/2204.03916/) (2022)
- [Efficient Architecture Search via Bi-level Data Pruning](https://mlsanity.com/paper/2312.14200/) (2023)
- [A Pairwise Comparison Relation-assisted Multi-objective Evolutionary Neural Architecture Search Method with Multi-population Mechanism](https://mlsanity.com/paper/2407.15600/) (2024)
- [α DARTS Once More: Enhancing Differentiable Architecture Search by Masked Image Modeling](https://mlsanity.com/paper/2211.10105/) (2022)
- [EM-DARTS: Hierarchical Differentiable Architecture Search for Eye Movement Recognition](https://mlsanity.com/paper/2409.14432/) (2024)
- [Mutually-aware Sub-Graphs Differentiable Architecture Search](https://mlsanity.com/paper/2107.04324/) (2021)
- [ICDARTS: Improving the Stability and Performance of Cyclic DARTS](https://mlsanity.com/paper/2309.00664/) (2023)
- [MGAS: Multi-Granularity Architecture Search for Trade-Off Between Model Effectiveness and Efficiency](https://mlsanity.com/paper/2310.15074/) (2023)
- [emoDARTS: Joint Optimisation of CNN & Sequential Neural Network Architectures for Superior Speech Emotion Recognition](https://mlsanity.com/paper/2403.14083/) (2024)
- [FX-DARTS: Designing Topology-unconstrained Architectures with Differentiable Architecture Search and Entropy-based Super-network Shrinking](https://mlsanity.com/paper/2504.20079/) (2025)
- [DNAD: Differentiable Neural Architecture Distillation](https://mlsanity.com/paper/2504.20080/) (2025)

## Abstract (arXiv)

Differentiable ARchiTecture Search, i.e., DARTS, has drawn great attention in neural architecture search. It tries to find the optimal architecture in a shallow search network and then measures its performance in a deep evaluation network. The independent optimization of the search and evaluation networks, however, leaves room for potential improvement by allowing interaction between the two networks. To address the problematic optimization issue, we propose new joint optimization objectives and a novel Cyclic Differentiable ARchiTecture Search framework, dubbed CDARTS. Considering the structure difference, CDARTS builds a cyclic feedback mechanism between the search and evaluation networks with introspective distillation. First, the search network generates an initial architecture for evaluation, and the weights of the evaluation network are optimized. Second, the architecture weights in the search network are further optimized by the label supervision in classification, as well as the regularization from the evaluation network through feature distillation. Repeating the above cycle results in joint optimization of the search and evaluation networks and thus enables the evolution of the architecture to fit the final evaluation network. The experiments and analysis on CIFAR, ImageNet and NAS-Bench-201 demonstrate the effectiveness of the proposed approach over the state-of-the-art ones. Specifically, in the DARTS search space, we achieve 97.52% top-1 accuracy on CIFAR10 and 76.3% top-1 accuracy on ImageNet. In the chain-structured search space, we achieve 78.2% top-1 accuracy on ImageNet, which is 1.1% higher than EfficientNet-B0. Our code and models are publicly available at https://github.com/microsoft/Cream.