# Reinforcement learning based adaptive metaheuristics

arXiv: https://arxiv.org/abs/2206.12233 · HTML: https://mlsanity.com/paper/2206.12233/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 37 references that are in the map.
- First posted on arXiv 24 Jun 2022.

## Builds on (most cited first, 5 of 5)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [Addressing Function Approximation Error in Actor-Critic Methods](https://mlsanity.com/paper/1802.09477/) (2018)
- [Deep Reinforcement Learning Based Parameter Control in Differential Evolution](https://mlsanity.com/paper/1905.08006/) (2019)
- [Learning adaptive differential evolution algorithm from optimization experiences by policy gradient](https://mlsanity.com/paper/2102.03572/) (2021)

## Built on by (most cited first, 7 of 7)

- [Evolutionary Optimization of Physics-Informed Neural Networks: Evo-PINN Frontiers and Opportunities](https://mlsanity.com/paper/2501.06572/) (2025)
- [Reinforcement Learning-assisted Evolutionary Algorithm: A Survey and Research Opportunities](https://mlsanity.com/paper/2308.13420/) (2023)
- [Using Automated Algorithm Configuration for Parameter Control](https://mlsanity.com/paper/2302.12334/) (2023)
- [AutoEP: LLMs-Driven Automation of Hyperparameter Evolution for Metaheuristic Algorithms](https://mlsanity.com/paper/2509.23189/) (2025)
- [Multi-parameter Control for the (1+(λ,λ))-GA on OneMax via Deep Reinforcement Learning](https://mlsanity.com/paper/2505.12982/) (2025)
- [RAG/LLM Augmented Switching Driven Polymorphic Metaheuristic Framework](https://mlsanity.com/paper/2505.13808/) (2025)
- [An exploration for higher efficiency in multi objective optimisation with reinforcement learning](https://mlsanity.com/paper/2512.10208/) (2025)

## Abstract (arXiv)

Parameter adaptation, that is the capability to automatically adjust an algorithm's hyperparameters depending on the problem being faced, is one of the main trends in evolutionary computation applied to numerical optimization. While several handcrafted adaptation policies have been proposed over the years to address this problem, only few attempts have been done so far at applying machine learning to learn such policies. Here, we introduce a general-purpose framework for performing parameter adaptation in continuous-domain metaheuristics based on state-of-the-art reinforcement learning algorithms. We demonstrate the applicability of this framework on two algorithms, namely Covariance Matrix Adaptation Evolution Strategies (CMA-ES) and Differential Evolution (DE), for which we learn, respectively, adaptation policies for the step-size (for CMA-ES), and the scale factor and crossover rate (for DE). We train these policies on a set of 46 benchmark functions at different dimensionalities, with various inputs to the policies, in two settings: one policy per function, and one global policy for all functions. Compared, respectively, to the Cumulative Step-size Adaptation (CSA) policy and to two well-known adaptive DE variants (iDE and jDE), our policies are able to produce competitive results in the majority of cases, especially in the case of DE.