mlsanityData to 14 Oct 2026

Differentiable Architecture Search for Reinforcement Learning

arXiv 2106.02229 ↗First posted 4 Jun 2021cs.LG cs.AI cs.CVcited by 4

Not on the map. Why?

Abstract (arXiv)

In this paper, we investigate the fundamental question: To what extent are gradient-based neural architecture search (NAS) techniques applicable to RL? Using the original DARTS as a convenient baseline, we discover that the discrete architectures found can achieve up to 250% performance compared to manual architecture designs on both discrete and continuous action space environments across off-policy and on-policy RL algorithms, at only 3x more computation time. Furthermore, through numerous ablation studies, we systematically verify that not only does DARTS correctly upweight operations during its supernet phrase, but also gradually improves resulting discrete cells up to 30x more efficiently than random search, suggesting DARTS is surprisingly an effective tool for improving architectures in RL.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 47 of its 79 references that are in the map. First posted on arXiv 4 Jun 2021.

Builds on

47 of 79 references.

References · 47

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  3. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  4. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  5. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,920
  6. 2016Neural Architecture Search with Reinforcement Learningcited by 1,499
  7. 2017Learning Transferable Architectures for Scalable Image Recognitioncited by 1,101
  8. 2015Prioritized Experience Replaycited by 1,099
  9. 2018DeepMind Control Suitecited by 956
  10. 2018Regularized Evolution for Image Classifier Architecture Searchcited by 909
  11. 2018IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architecturescited by 883
  12. 2018Efficient Neural Architecture Search via Parameter Sharingcited by 837
35 more
  1. 2017Rainbow: Combining Improvements in Deep Reinforcement Learningcited by 786
  2. 2017A Distributional Perspective on Reinforcement Learningcited by 765
  3. 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 743
  4. 2018ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardwarecited by 673
  5. 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 585
  6. 2017Evolution Strategies as a Scalable Alternative to Reinforcement Learningcited by 574
  7. 2016RL2: Fast Reinforcement Learning via Slow Reinforcement Learningcited by 503
  8. 2017Progressive Neural Architecture Searchcited by 487
  9. 2019Leveraging Procedural Generation to Benchmark Reinforcement Learningcited by 483
  10. 2020NAS-Bench-201: Extending the Scope of Reproducible Neural Architecture Searchcited by 416
  11. 2016Learning to reinforcement learncited by 377
  12. 2018Quantifying Generalization in Reinforcement Learningcited by 352
  13. 2019Searching for A Robust Neural Architecture in Four GPU Hourscited by 260
  14. 2020Neural Architecture Search without Trainingcited by 244
  15. 2019Understanding and Robustifying Differentiable Architecture Searchcited by 178
  16. 2018Sim-to-Real via Sim-to-Sim: Data-efficient Robotic Grasping via Randomized-to-Canonical Adaptation Networkscited by 174
  17. 2018Neural Architecture Optimizationcited by 167
  18. 2017Neural Episodic Controlcited by 151
  19. 2020Acme: A Research Framework for Distributed Reinforcement Learningcited by 146
  20. 2019Fair DARTS: Eliminating Unfair Advantages in Differentiable Architecture Searchcited by 131
  21. 2019DARTS+: Improved Differentiable Architecture Search with Early Stoppingcited by 105
  22. 2020Stabilizing Differentiable Architecture Search via Perturbation-based Regularizationcited by 102
  23. 2020RL-CycleGAN: Reinforcement Learning Aware Simulation-To-Realcited by 91
  24. 2021How Powerful are Performance Predictors in Neural Architecture Search?cited by 90
  25. 2021Decoupling Value and Policy for Generalization in Reinforcement Learningcited by 82
  26. 2020DARTS-: Robustly Stepping out of Performance Collapse Without Indicatorscited by 81
  27. 2019Weight Agnostic Neural Networkscited by 71
  28. 2020Automatic Data Augmentation for Generalization in Deep Reinforcement Learningcited by 71
  29. 2019SGAS: Sequential Greedy Architecture Searchcited by 69
  30. 2020Provably Efficient Online Hyperparameter Optimization with Population-Based Banditscited by 63
  31. 2021On the Importance of Hyperparameter Optimization for Model-based Reinforcement Learningcited by 59
  32. 2020Geometry-Aware Gradient Algorithms for Neural Architecture Searchcited by 40
  33. 2019Generalization of Reinforcement Learners with Working and Episodic Memorycited by 36
  34. 2019sharpDARTS: Faster and More Accurate Differentiable Architecture Searchcited by 19
  35. 2021Visionary: Vision architecture discovery for robot learningcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 1; 2023: 2; 2024: 0; 2025: 0; 2026: 1.

4 papers.

  1. 2023Differentiable Quantum Architecture Search for Quantum Reinforcement Learningcited by 13
  2. 2022Automated Reinforcement Learning: An Overviewcited by 9
  3. 2023HyperPPO: A scalable method for finding small policies for robotic controlcited by 4
  4. 2026Temporal Attention for Adaptive Control of Euler-Lagrange Systems with Unobservable Memorycited by 1