# Neural Architecture Evolution in Deep Reinforcement Learning for Continuous Control

arXiv: https://arxiv.org/abs/1910.12824 · HTML: https://mlsanity.com/paper/1910.12824/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Oct 2019.

## Built on by (most cited first, 4 of 4)

- [Evolutionary Reinforcement Learning: A Survey](https://mlsanity.com/paper/2303.04150/) (2023)
- [Behavior-based Neuroevolutionary Training in Reinforcement Learning](https://mlsanity.com/paper/2105.07960/) (2021)
- [Evolutionary Discovery of Reinforcement Learning Algorithms via Large Language Models](https://mlsanity.com/paper/2603.28416/) (2026)
- [Small Data Explainer -- The impact of small data methods in everyday life](https://mlsanity.com/paper/2507.11773/) (2025)

## Abstract (arXiv)

Current Deep Reinforcement Learning algorithms still heavily rely on handcrafted neural network architectures. We propose a novel approach to automatically find strong topologies for continuous control tasks while only adding a minor overhead in terms of interactions in the environment. To achieve this, we combine Neuroevolution techniques with off-policy training and propose a novel architecture mutation operator. Experiments on five continuous control benchmarks show that the proposed Actor-Critic Neuroevolution algorithm often outperforms the strong Actor-Critic baseline and is capable of automatically finding topologies in a sample-efficient manner which would otherwise have to be found by expensive architecture search.