# Bridging Evolutionary Algorithms and Reinforcement Learning: A Comprehensive Survey on Hybrid Algorithms

arXiv: https://arxiv.org/abs/2401.11963 · HTML: https://mlsanity.com/paper/2401.11963/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- Placed in RL for games and combinatorial optimization, one of 15 directions in Deep reinforcement learning methods.
- Builds on 113 of its 286 references that are in the map.
- First posted on arXiv 22 Jan 2024.

## Placed in

- Deep reinforcement learning methods / [RL for games and combinatorial optimization](https://mlsanity.com/deeprl/rl-for-games-and-combinatorial-optimization/)

## Builds on (most cited first, 30 of 113)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Trust Region Policy Optimization](https://mlsanity.com/paper/1502.05477/) (2015)
- [Addressing Function Approximation Error in Actor-Critic Methods](https://mlsanity.com/paper/1802.09477/) (2018)
- [Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments](https://mlsanity.com/paper/1706.02275/) (2017)
- [Learning Latent Dynamics for Planning from Pixels](https://mlsanity.com/paper/1811.04551/) (2018)
- [Exploration by Random Network Distillation](https://mlsanity.com/paper/1810.12894/) (2018)
- [DeepMind Control Suite](https://mlsanity.com/paper/1801.00690/) (2018)
- [Recurrent World Models Facilitate Policy Evolution](https://mlsanity.com/paper/1809.01999/) (2018)
- [Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Models](https://mlsanity.com/paper/1805.12114/) (2018)
- [Diversity is All You Need: Learning Skills without a Reward Function](https://mlsanity.com/paper/1802.06070/) (2018)
- [QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulation](https://mlsanity.com/paper/1806.10293/) (2018)
- [QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1803.11485/) (2018)
- [Learning to learn by gradient descent by gradient descent](https://mlsanity.com/paper/1606.04474/) (2016)
- [The StarCraft Multi-Agent Challenge](https://mlsanity.com/paper/1902.04043/) (2019)
- [Evolution Strategies as a Scalable Alternative to Reinforcement Learning](https://mlsanity.com/paper/1703.03864/) (2017)
- [Eureka: Human-Level Reward Design via Coding Large Language Models](https://mlsanity.com/paper/2310.12931/) (2023)
- [Reinforcement Learning with Augmented Data](https://mlsanity.com/paper/2004.14990/) (2020)
- [Illuminating search spaces by mapping elites](https://mlsanity.com/paper/1504.04909/) (2015)
- [Temporal Difference Learning for Model Predictive Control](https://mlsanity.com/paper/2203.04955/) (2022)
- [Population Based Training of Neural Networks](https://mlsanity.com/paper/1711.09846/) (2017)
- [Randomized Ensembled Double Q-Learning: Learning Fast Without a Model](https://mlsanity.com/paper/2101.05982/) (2021)
- [Residual Reinforcement Learning for Robot Control](https://mlsanity.com/paper/1812.03201/) (2018)
- [The Primacy Bias in Deep Reinforcement Learning](https://mlsanity.com/paper/2205.07802/) (2022)
- [Reinforcement Learning for Combinatorial Optimization: A Survey](https://mlsanity.com/paper/2003.03600/) (2020)
- [Go-Explore: a New Approach for Hard-Exploration Problems](https://mlsanity.com/paper/1901.10995/) (2019)
- [FACMAC: Factored Multi-Agent Centralised Policy Gradients](https://mlsanity.com/paper/2003.06709/) (2020)
- [Controlling Overestimation Bias with Truncated Mixture of Continuous Distributional Quantile Critics](https://mlsanity.com/paper/2005.04269/) (2020)

## Built on by (most cited first, 26 of 26)

- [Toward Automated Algorithm Design: A Survey and Practical Guide to Meta-Black-Box-Optimization](https://mlsanity.com/paper/2411.00625/) (2024)
- [The impact of intrinsic rewards on exploration in Reinforcement Learning](https://mlsanity.com/paper/2501.11533/) (2025)
- [Advancing CMA-ES with Learning-Based Cooperative Coevolution for Scalable Optimization](https://mlsanity.com/paper/2504.17578/) (2025)
- [Embodied Co-Design for Rapidly Evolving Agents: Taxonomy, Frontiers, and Challenges](https://mlsanity.com/paper/2512.04770/) (2025)
- [Accelerating Vehicle Routing via AI-Initialized Genetic Algorithms](https://mlsanity.com/paper/2504.06126/) (2025)
- [An Efficient Task-Oriented Dialogue Policy: Evolutionary Reinforcement Learning Injected by Elite Individuals](https://mlsanity.com/paper/2506.03519/) (2025)
- [The Price equation reveals a universal force-metric-bias law of algorithmic learning and natural selection](https://mlsanity.com/paper/2507.18549/) (2025)
- [Scaling DRL for Decision Making: A Survey on Data, Network, and Training Budget Strategies](https://mlsanity.com/paper/2508.03194/) (2025)
- [Meta-Learning-Assisted Constraint Relaxation for Constrained Black-Box Optimization](https://mlsanity.com/paper/2602.00532/) (2026)
- [Deep Reinforcement Learning-Assisted Automated Operator Portfolio for Constrained Multi-objective Optimization](https://mlsanity.com/paper/2603.16401/) (2026)
- [A Learning-Based Cooperative Coevolution Framework for Heterogeneous Large-Scale Global Optimization](https://mlsanity.com/paper/2604.01241/) (2026)
- [Evolutionary Enhanced Multi-Agent Reinforcement Learning for Cooperative Air Combat](https://mlsanity.com/paper/2605.25091/) (2026)
- [Reinforcement learning Based Automated Design of Differential Evolution Algorithm for Black-box Optimization](https://mlsanity.com/paper/2501.12881/) (2025)
- [Physics-Informed Evolution: An Evolutionary Framework for Solving Quantum Control Problems Involving the Schr"odinger Equation](https://mlsanity.com/paper/2502.05228/) (2025)
- [Unlearning Works Better Than You Think: Local Reinforcement-Based Selection of Auxiliary Objectives](https://mlsanity.com/paper/2504.14418/) (2025)
- [Randomised Optimism via Competitive Co-Evolution for Matrix Games with Bandit Feedback](https://mlsanity.com/paper/2505.13562/) (2025)
- [A Study of Hybrid and Evolutionary Metaheuristics for Single Hidden Layer Feedforward Neural Network Architecture](https://mlsanity.com/paper/2506.15737/) (2025)
- [An In-depth Study of LLM Contributions to the Bin Packing Problem](https://mlsanity.com/paper/2510.27353/) (2025)
- [CogMCTS: A Novel Cognitive-Guided Monte Carlo Tree Search Framework for Iterative Heuristic Evolution with Large Language Models](https://mlsanity.com/paper/2512.08609/) (2025)
- [READY: Reward Discovery for Meta-Black-Box Optimization](https://mlsanity.com/paper/2601.21847/) (2026)
- [Decoupling Numerical and Structural Parameters: An Empirical Study on Adaptive Genetic Algorithms via Deep Reinforcement Learning for the Large-Scale TSP](https://mlsanity.com/paper/2603.20702/) (2026)
- [Evolution-Inspired Sample Competition for Deep Neural Network Optimization](https://mlsanity.com/paper/2604.12568/) (2026)
- [Contextual Quality-Diversity Evolutionary Reinforcement Learning for HVAC Control in Tropical Commercial Buildings](https://mlsanity.com/paper/2608.11324/) (2026)
- [Emergent Charging Coordination in Electric Delivery Fleets](https://mlsanity.com/paper/2609.07689/) (2026)
- [EVOL: Simulator-Guided Evolutionary Expert Synthesis for Deployment-Free Learning Path Recommendation](https://mlsanity.com/paper/2610.03273/) (2026)
- [Learning to Orchestrate Evolutionary Search: Progression-Aware Deep Reinforcement Learning for Dynamic DE-CMA-ES Coordination in Optimization and Structural Model Updating](https://mlsanity.com/paper/2610.11546/) (2026)

## Abstract (arXiv)

Evolutionary Reinforcement Learning (ERL), which integrates Evolutionary Algorithms (EAs) and Reinforcement Learning (RL) for optimization, has demonstrated remarkable performance advancements. By fusing both approaches, ERL has emerged as a promising research direction. This survey offers a comprehensive overview of the diverse research branches in ERL. Specifically, we systematically summarize recent advancements in related algorithms and identify three primary research directions: EA-assisted Optimization of RL, RL-assisted Optimization of EA, and synergistic optimization of EA and RL. Following that, we conduct an in-depth analysis of each research direction, organizing multiple research branches. We elucidate the problems that each branch aims to tackle and how the integration of EAs and RL addresses these challenges. In conclusion, we discuss potential challenges and prospective future research directions across various research directions. To facilitate researchers in delving into ERL, we organize the algorithms and codes involved on https://github.com/yeshenpy/Awesome-Evolutionary-Reinforcement-Learning.