# Reinforcement Learning-based Self-adaptive Differential Evolution through Automated Landscape Feature Learning

arXiv: https://arxiv.org/abs/2503.18061 · HTML: https://mlsanity.com/paper/2503.18061/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Placed in RL for games and combinatorial optimization, one of 15 directions in Deep reinforcement learning methods.
- Builds on 15 of its 75 references that are in the map.
- First posted on arXiv 23 Mar 2025.

## Placed in

- Deep reinforcement learning methods / [RL for games and combinatorial optimization](https://mlsanity.com/deeprl/rl-for-games-and-combinatorial-optimization/)

## Builds on (most cited first, 15 of 15)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Deep Reinforcement Learning with Double Q-learning](https://mlsanity.com/paper/1509.06461/) (2015)
- [LLaMoCo: Instruction Tuning of Large Language Models for Optimization Code Generation](https://mlsanity.com/paper/2403.01131/) (2024)
- [Deep Reinforcement Learning Based Parameter Control in Differential Evolution](https://mlsanity.com/paper/1905.08006/) (2019)
- [MetaBox: A Benchmark Platform for Meta-Black-Box Optimization with Reinforcement Learning](https://mlsanity.com/paper/2310.08252/) (2023)
- [Deep Reinforcement Learning for Dynamic Algorithm Selection: A Proof-of-Principle Study on Differential Evolution](https://mlsanity.com/paper/2403.02131/) (2024)
- [Symbol: Generating Flexible Black-Box Optimizers through Symbolic Equation Learning](https://mlsanity.com/paper/2402.02355/) (2024)
- [Auto-configuring Exploration-Exploitation Tradeoff in Evolutionary Computation via Deep Reinforcement Learning](https://mlsanity.com/paper/2404.08239/) (2024)
- [B2Opt: Learning to Optimize Black-box Optimization with Little Budget](https://mlsanity.com/paper/2304.11787/) (2023)
- [Learning from Offline and Online Experiences: A Hybrid Adaptive Operator Selection Framework](https://mlsanity.com/paper/2404.10252/) (2024)
- [Automated Metaheuristic Algorithm Design with Autoregressive Learning](https://mlsanity.com/paper/2405.03419/) (2024)
- [Learning to Transfer for Evolutionary Multitasking](https://mlsanity.com/paper/2406.14359/) (2024)

## Built on by (most cited first, 12 of 12)

- [MetaBox-v2: A Unified Benchmark Platform for Meta-Black-Box Optimization](https://mlsanity.com/paper/2505.17745/) (2025)
- [Instance Generation for Meta-Black-Box Optimization through Latent Space Reverse Engineering](https://mlsanity.com/paper/2509.15810/) (2025)
- [DesignX: Human-Competitive Algorithm Designer for Black-Box Optimization](https://mlsanity.com/paper/2505.17866/) (2025)
- [Advancing CMA-ES with Learning-Based Cooperative Coevolution for Scalable Optimization](https://mlsanity.com/paper/2504.17578/) (2025)
- [Task-free Adaptive Meta Black-box Optimization](https://mlsanity.com/paper/2601.21475/) (2026)
- [Learning Where, What and How to Transfer: A Multi-Role Reinforcement Learning Approach for Evolutionary Multitasking](https://mlsanity.com/paper/2511.15199/) (2025)
- [Deep Reinforcement Learning for Dynamic Algorithm Configuration: A Case Study on Optimizing OneMax with the (1+(λ,λ))-GA](https://mlsanity.com/paper/2512.03805/) (2025)
- [Detect and Act: Automated Dynamic Optimizer through Meta-Black-Box Optimization](https://mlsanity.com/paper/2601.22542/) (2026)
- [Meta-Learning-Assisted Constraint Relaxation for Constrained Black-Box Optimization](https://mlsanity.com/paper/2602.00532/) (2026)
- [READY: Reward Discovery for Meta-Black-Box Optimization](https://mlsanity.com/paper/2601.21847/) (2026)
- [Surrogate Ensemble in Expensive Multi-Objective Optimization via Deep Q-Learning](https://mlsanity.com/paper/2602.00540/) (2026)
- [Discovering Interpretable Multi-Parameter Control Policies for Evolutionary Algorithms Using Deep Reinforcement Learning](https://mlsanity.com/paper/2606.10129/) (2026)

## Abstract (arXiv)

Recently, Meta-Black-Box-Optimization (MetaBBO) methods significantly enhance the performance of traditional black-box optimizers through meta-learning flexible and generalizable meta-level policies that excel in dynamic algorithm configuration (DAC) tasks within the low-level optimization, reducing the expertise required to adapt optimizers for novel optimization tasks. Though promising, existing MetaBBO methods heavily rely on human-crafted feature extraction approach to secure learning effectiveness. To address this issue, this paper introduces a novel MetaBBO method that supports automated feature learning during the meta-learning process, termed as RLDE-AFL, which integrates a learnable feature extraction module into a reinforcement learning-based DE method to learn both the feature encoding and meta-level policy. Specifically, we design an attention-based neural network with mantissa-exponent based embedding to transform the solution populations and corresponding objective values during the low-level optimization into expressive landscape features. We further incorporate a comprehensive algorithm configuration space including diverse DE operators into a reinforcement learning-aided DAC paradigm to unleash the behavior diversity and performance of the proposed RLDE-AFL. Extensive benchmark results show that co-training the proposed feature learning module and DAC policy contributes to the superior optimization performance of RLDE-AFL to several advanced DE methods and recent MetaBBO baselines over both synthetic and realistic BBO scenarios. The source codes of RLDE-AFL are available at https://github.com/GMC-DRL/RLDE-AFL.