# When Does Neuroevolution Outcompete Reinforcement Learning in Transfer Learning Tasks?

arXiv: https://arxiv.org/abs/2505.22696 · HTML: https://mlsanity.com/paper/2505.22696/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods.
- Builds on 18 of its 51 references that are in the map.
- First posted on arXiv 28 May 2025.

## Placed in

- Deep reinforcement learning methods / [Deep RL algorithms and representations](https://mlsanity.com/deeprl/deep-rl-algorithms-and-representations/)

## Builds on (most cited first, 18 of 18)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Diversity is All You Need: Learning Skills without a Reward Function](https://mlsanity.com/paper/1802.06070/) (2018)
- [Evolution Strategies as a Scalable Alternative to Reinforcement Learning](https://mlsanity.com/paper/1703.03864/) (2017)
- [Brax -- A Differentiable Physics Engine for Large Scale Rigid Body Simulation](https://mlsanity.com/paper/2106.13281/) (2021)
- [The CMA Evolution Strategy: A Tutorial](https://mlsanity.com/paper/1604.00772/) (2016)
- [First return, then explore](https://mlsanity.com/paper/2004.12919/) (2020)
- [Deep Neuroevolution: Genetic Algorithms Are a Competitive Alternative for Training Deep Neural Networks for Reinforcement Learning](https://mlsanity.com/paper/1712.06567/) (2017)
- [Autotelic Agents with Intrinsically Motivated Goal-Conditioned Reinforcement Learning: a Short Survey](https://mlsanity.com/paper/2012.09830/) (2020)
- [Continual Model-Based Reinforcement Learning with Hypernetworks](https://mlsanity.com/paper/2009.11997/) (2020)
- [Neuroevolution is a Competitive Alternative to Reinforcement Learning for Skill Discovery](https://mlsanity.com/paper/2210.03516/) (2022)
- [HyperNCA: Growing Developmental Networks with Neural Cellular Automata](https://mlsanity.com/paper/2204.11674/) (2022)
- [QDax: A Library for Quality-Diversity and Population-based Algorithms with Hardware Acceleration](https://mlsanity.com/paper/2308.03665/) (2023)
- [Accelerated Quality-Diversity through Massive Parallelism](https://mlsanity.com/paper/2202.01258/) (2022)
- [Tensorized NeuroEvolution of Augmenting Topologies for GPU Acceleration](https://mlsanity.com/paper/2404.01817/) (2024)
- [Understanding fitness landscapes in morpho-evolution via local optima networks](https://mlsanity.com/paper/2402.07822/) (2024)
- [Growing Artificial Neural Networks for Control: the Role of Neuronal Diversity](https://mlsanity.com/paper/2405.08510/) (2024)
- [Articulated Animal AI: An Environment for Animal-like Cognition in a Limbed Agent](https://mlsanity.com/paper/2410.09275/) (2024)

## Built on by (most cited first, 2 of 2)

- [Distributional Value Estimation Without Target Networks for Robust Quality-Diversity](https://mlsanity.com/paper/2604.20381/) (2026)
- [Continual Reinforcement Learning with Neuroevolution](https://mlsanity.com/paper/2610.01583/) (2026)

## Abstract (arXiv)

The ability to continuously and efficiently transfer skills across tasks is a hallmark of biological intelligence and a long-standing goal in artificial systems. Reinforcement learning (RL), a dominant paradigm for learning in high-dimensional control tasks, is known to suffer from brittleness to task variations and catastrophic forgetting. Neuroevolution (NE) has recently gained attention for its robustness, scalability, and capacity to escape local optima. In this paper, we investigate an understudied dimension of NE: its transfer learning capabilities. To this end, we introduce two benchmarks: a) in stepping gates, neural networks are tasked with emulating logic circuits, with designs that emphasize modular repetition and variation b) ecorobot extends the Brax physics engine with objects such as walls and obstacles and the ability to easily switch between different robotic morphologies. Crucial in both benchmarks is the presence of a curriculum that enables evaluating skill transfer across tasks of increasing complexity. Our empirical analysis shows that NE methods vary in their transfer abilities and frequently outperform RL baselines. Our findings support the potential of NE as a foundation for building more adaptable agents and highlight future challenges for scaling NE to complex, real-world problems.