mlsanityData to 7 Oct 2026

RADARS: Memory Efficient Reinforcement Learning Aided Differentiable Neural Architecture Search

arXiv 2109.05691 ↗First posted 13 Sep 2021cs.LGcited by 8

Not on the map. Why?

Abstract (arXiv)

Differentiable neural architecture search (DNAS) is known for its capacity in the automatic generation of superior neural networks. However, DNAS based methods suffer from memory usage explosion when the search space expands, which may prevent them from running successfully on even advanced GPU platforms. On the other hand, reinforcement learning (RL) based methods, while being memory efficient, are extremely time-consuming. Combining the advantages of both types of methods, this paper presents RADARS, a scalable RL-aided DNAS framework that can explore large search spaces in a fast and memory-efficient manner. RADARS iteratively applies RL to prune undesired architecture candidates and identifies a promising subspace to carry out DNAS. Experiments using a workstation with 12 GB GPU memory show that on CIFAR-10 and ImageNet datasets, RADARS can achieve up to 3.41% higher accuracy with 2.5X search time reduction compared with a state-of-the-art RL-based method, while the two DNAS baselines cannot complete due to excessive memory usage or search time. To the best of the authors' knowledge, this is the first DNAS framework that can handle large search spaces with bounded memory usage.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 18 of its 20 references that are in the map. First posted on arXiv 13 Sep 2021.

Builds on

18 of 20 references.

References · 18

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
  3. 2018DARTS: Differentiable Architecture Searchcited by 1,782
  4. 2016Neural Architecture Search with Reinforcement Learningcited by 1,489
  5. 2017Learning Transferable Architectures for Scalable Image Recognitioncited by 1,097
  6. 2018ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardwarecited by 666
  7. 2017Progressive Neural Architecture Searchcited by 483
  8. 2017CondenseNet: An Efficient DenseNet using Learned Group Convolutionscited by 121
  9. 2019One-Shot Neural Architecture Search via Self-Evaluated Template Networkcited by 69
  10. 2019Hardware/Software Co-Exploration of Neural Architecturescited by 50
  11. 2020Few-shot Neural Architecture Searchcited by 50
  12. 2020Co-Exploration of Neural Architectures and Heterogeneous ASIC Accelerator Designs Targeting Multiple Taskscited by 43
6 more
  1. 2019Accuracy vs. Efficiency: Achieving Both through FPGA-Implementation Aware Neural Architecture Searchcited by 38
  2. 2020EDD: Efficient Differentiable DNN Architecture and Implementation Co-search for Embedded AI Solutionscited by 35
  3. 2019Device-Circuit-Architecture Co-Exploration for Computing-in-Memory Neural Acceleratorscited by 32
  4. 2019UNAS: Differentiable Architecture Search Meets Reinforcement Learningcited by 21
  5. 2019On Neural Architecture Search for Resource-Constrained Hardware Platformscited by 18
  6. 2021Uncertainty Modeling of Emerging Device-based Computing-in-Memory Neural Accelerators with Application to Neural Architecture Searchcited by 17

Built on by

Cited by year

Citing papers per year: 2022: 2; 2023: 5; 2024: 0; 2025: 1.

8 papers.

  1. 2022SWIM: Selective Write-Verify for Computing-in-Memory Neural Acceleratorscited by 16
  2. 2022Computing-In-Memory Neural Network Accelerators for Safety-Critical Systems: Can Small Device Variations Be Disastrous?cited by 12
  3. 2023RRNet: Towards ReLU-Reduced Neural Network for Two-party Computation Based Private Inferencecited by 12
  4. 2023Improving Realistic Worst-Case Performance of NVCiM DNN Accelerators through Training with Right-Censored Gaussian Noisecited by 8
  5. 2023Efficient Automation of Neural Network Design: A Survey on Differentiable Neural Architecture Searchcited by 5
  6. 2023Compute-in-Memory based Neural Network Accelerators for Safety-Critical Systems: Worst-Case Scenarios and Protectionscited by 2
  7. 2023U-SWIM: Universal Selective Write-Verify for Computing-in-Memory Neural Acceleratorscited by 0
  8. 2025An Integrated Approach to Neural Architecture Search for Deep Q-Networkscited by 0