# Deep Reinforcement Learning-Assisted Component Auto-Configuration of Differential Evolution Algorithm for Constrained Optimization: A Foundation Model

arXiv: https://arxiv.org/abs/2509.11016 · HTML: https://mlsanity.com/paper/2509.11016/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL for games and combinatorial optimization, one of 15 directions in Deep reinforcement learning methods.
- Builds on 9 of its 47 references that are in the map.
- First posted on arXiv 14 Sep 2025.

## Placed in

- Deep reinforcement learning methods / [RL for games and combinatorial optimization](https://mlsanity.com/deeprl/rl-for-games-and-combinatorial-optimization/)

## Builds on (most cited first, 9 of 9)

- [Deep Reinforcement Learning for Autonomous Driving: A Survey](https://mlsanity.com/paper/2002.00444/) (2020)
- [A Brief Survey of Deep Reinforcement Learning](https://mlsanity.com/paper/1708.05866/) (2017)
- [Applications of Deep Reinforcement Learning in Communications and Networking: A Survey](https://mlsanity.com/paper/1810.07862/) (2018)
- [Two-Archive Evolutionary Algorithm for Constrained Multi-Objective Optimization](https://mlsanity.com/paper/1711.07907/) (2017)
- [Deep Reinforcement Learning Control for Radar Detection and Tracking in Congested Spectral Environments](https://mlsanity.com/paper/2006.13173/) (2020)
- [Reconfigurable Intelligent Surfaces for Wireless Communications: Principles, Challenges, and Opportunities](https://mlsanity.com/paper/2005.00938/) (2020)
- [Principled Design of Translation, Scale, and Rotation Invariant Variation Operators for Metaheuristics](https://mlsanity.com/paper/2105.10657/) (2021)
- [Constrained Multi-objective Optimization with Deep Reinforcement Learning Assisted Operator Selection](https://mlsanity.com/paper/2402.12381/) (2024)
- [A Review on Constraint Handling Techniques for Population-based Algorithms: from single-objective to multi-objective optimization](https://mlsanity.com/paper/2206.13802/) (2022)

## Abstract (arXiv)

Despite significant efforts to manually design high-performance evolutionary algorithms, their adaptability remains limited due to the dynamic and ever-evolving nature of real-world problems. The "no free lunch" theorem highlights that no single algorithm performs optimally across all problems. While online adaptation methods have been proposed, they often suffer from inefficiency, weak convergence, and limited generalization on constrained optimization problems (COPs). To address these challenges, we introduce a novel framework for automated component configuration in Differential Evolution (DE) algorithm to address COPs, powered by Deep Reinforcement Learning (DRL). Specifically, we propose SuperDE, a foundation model that dynamically configures DE's evolutionary components based on real-time evolution. Trained offline through meta-learning across a wide variety of COPs, SuperDE is capable of recommending optimal per-generation configurations for unseen problems in a zero-shot manner. Utilizing a Double Deep Q-Network (DDQN), SuperDE adapts its configuration strategies in response to the evolving population states during optimization. Experimental results demonstrate that SuperDE significantly outperforms existing state-of-the-art algorithms on benchmark test suites, achieving superior generalization and optimization performance.