# RelativeNAS: Relative Neural Architecture Search via Slow-Fast Learning

arXiv: https://arxiv.org/abs/2009.06193 · HTML: https://mlsanity.com/paper/2009.06193/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Sep 2020.

## Built on by (most cited first, 10 of 10)

- [Neural Architecture Search as Multiobjective Optimization Benchmarks: Problem Formulation and Performance Assessment](https://mlsanity.com/paper/2208.04321/) (2022)
- [A Survey on Computationally Efficient Neural Architecture Search](https://mlsanity.com/paper/2206.01520/) (2022)
- [Contrastive Self-supervised Neural Architecture Search](https://mlsanity.com/paper/2102.10557/) (2021)
- [Accelerating Multi-Objective Neural Architecture Search by Random-Weight Evaluation](https://mlsanity.com/paper/2110.05242/) (2021)
- [Designing Novel Cognitive Diagnosis Models via Evolutionary Multi-Objective Neural Architecture Search](https://mlsanity.com/paper/2307.04429/) (2023)
- [A Pairwise Comparison Relation-assisted Multi-objective Evolutionary Neural Architecture Search Method with Multi-population Mechanism](https://mlsanity.com/paper/2407.15600/) (2024)
- [TND-NAS: Towards Non-differentiable Objectives in Progressive Differentiable NAS Framework](https://mlsanity.com/paper/2111.03892/) (2021)
- [FR-NAS: Forward-and-Reverse Graph Predictor for Efficient Neural Architecture Search](https://mlsanity.com/paper/2404.15622/) (2024)
- [FX-DARTS: Designing Topology-unconstrained Architectures with Differentiable Architecture Search and Entropy-based Super-network Shrinking](https://mlsanity.com/paper/2504.20079/) (2025)
- [DNAD: Differentiable Neural Architecture Distillation](https://mlsanity.com/paper/2504.20080/) (2025)

## Abstract (arXiv)

Despite the remarkable successes of Convolutional Neural Networks (CNNs) in computer vision, it is time-consuming and error-prone to manually design a CNN. Among various Neural Architecture Search (NAS) methods that are motivated to automate designs of high-performance CNNs, the differentiable NAS and population-based NAS are attracting increasing interests due to their unique characters. To benefit from the merits while overcoming the deficiencies of both, this work proposes a novel NAS method, RelativeNAS. As the key to efficient search, RelativeNAS performs joint learning between fast-learners (i.e. networks with relatively higher accuracy) and slow-learners in a pairwise manner. Moreover, since RelativeNAS only requires low-fidelity performance estimation to distinguish each pair of fast-learner and slow-learner, it saves certain computation costs for training the candidate architectures. The proposed RelativeNAS brings several unique advantages: (1) it achieves state-of-the-art performance on ImageNet with top-1 error rate of 24.88%, i.e. outperforming DARTS and AmoebaNet-B by 1.82% and 1.12% respectively; (2) it spends only nine hours with a single 1080Ti GPU to obtain the discovered cells, i.e. 3.75x and 7875x faster than DARTS and AmoebaNet respectively; (3) it provides that the discovered cells obtained on CIFAR-10 can be directly transferred to object detection, semantic segmentation, and keypoint detection, yielding competitive results of 73.1% mAP on PASCAL VOC, 78.7% mIoU on Cityscapes, and 68.5% AP on MSCOCO, respectively. The implementation of RelativeNAS is available at https://github.com/EMI-Group/RelativeNAS