# Advancing CMA-ES with Learning-Based Cooperative Coevolution for Scalable Optimization

arXiv: https://arxiv.org/abs/2504.17578 · HTML: https://mlsanity.com/paper/2504.17578/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in RL for games and combinatorial optimization, one of 15 directions in Deep reinforcement learning methods.
- Builds on 27 of its 86 references that are in the map.
- First posted on arXiv 24 Apr 2025.

## Placed in

- Deep reinforcement learning methods / [RL for games and combinatorial optimization](https://mlsanity.com/deeprl/rl-for-games-and-combinatorial-optimization/)

## Builds on (most cited first, 27 of 27)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [A Survey Of Methods For Explaining Black Box Models](https://mlsanity.com/paper/1802.01933/) (2018)
- [The CMA Evolution Strategy: A Tutorial](https://mlsanity.com/paper/1604.00772/) (2016)
- [Deep Neuroevolution: Genetic Algorithms Are a Competitive Alternative for Training Deep Neural Networks for Reinforcement Learning](https://mlsanity.com/paper/1712.06567/) (2017)
- [Unbiased Gradient Estimation in Unrolled Computation Graphs with Persistent Evolution Strategies](https://mlsanity.com/paper/2112.13835/) (2021)
- [Discovering Evolution Strategies via Meta-Black-Box Optimization](https://mlsanity.com/paper/2211.11260/) (2022)
- [Deep Reinforcement Learning Based Parameter Control in Differential Evolution](https://mlsanity.com/paper/1905.08006/) (2019)
- [Multi-agent Dynamic Algorithm Configuration](https://mlsanity.com/paper/2210.06835/) (2022)
- [MetaBox: A Benchmark Platform for Meta-Black-Box Optimization with Reinforcement Learning](https://mlsanity.com/paper/2310.08252/) (2023)
- [Neuroevolution in Deep Neural Networks: Current Trends and Future Challenges](https://mlsanity.com/paper/2006.05415/) (2020)
- [Learning adaptive differential evolution algorithm from optimization experiences by policy gradient](https://mlsanity.com/paper/2102.03572/) (2021)
- [Bridging Evolutionary Algorithms and Reinforcement Learning: A Comprehensive Survey on Hybrid Algorithms](https://mlsanity.com/paper/2401.11963/) (2024)
- [Deep Reinforcement Learning for Dynamic Algorithm Selection: A Proof-of-Principle Study on Differential Evolution](https://mlsanity.com/paper/2403.02131/) (2024)
- [Symbol: Generating Flexible Black-Box Optimizers through Symbolic Equation Learning](https://mlsanity.com/paper/2402.02355/) (2024)
- [Auto-configuring Exploration-Exploitation Tradeoff in Evolutionary Computation via Deep Reinforcement Learning](https://mlsanity.com/paper/2404.08239/) (2024)
- [B2Opt: Learning to Optimize Black-box Optimization with Little Budget](https://mlsanity.com/paper/2304.11787/) (2023)
- [ConfigX: Modular Configuration for Evolutionary Algorithms via Multitask Reinforcement Learning](https://mlsanity.com/paper/2412.07507/) (2024)
- [Pretrained Optimization Model for Zero-Shot Black Box Optimization](https://mlsanity.com/paper/2405.03728/) (2024)
- [Reinforcement Learning-based Self-adaptive Differential Evolution through Automated Landscape Feature Learning](https://mlsanity.com/paper/2503.18061/) (2025)
- [MMES: Mixture Model based Evolution Strategy for Large-Scale Optimization](https://mlsanity.com/paper/2203.12675/) (2022)
- [MetaDE: Evolving Differential Evolution by Differential Evolution](https://mlsanity.com/paper/2502.10470/) (2025)
- [RLEMMO: Evolutionary Multimodal Optimization Assisted By Deep Reinforcement Learning](https://mlsanity.com/paper/2404.08242/) (2024)
- [Surrogate Learning in Meta-Black-Box Optimization: A Preliminary Study](https://mlsanity.com/paper/2503.18060/) (2025)
- [LM-CMA: an Alternative to L-BFGS for Large Scale Black-box Optimization](https://mlsanity.com/paper/1511.00221/) (2015)
- [Accurate Peak Detection in Multimodal Optimization via Approximated Landscape Learning](https://mlsanity.com/paper/2503.18066/) (2025)
- [An Enhanced Differential Grouping Method for Large-Scale Overlapping Problems](https://mlsanity.com/paper/2404.10515/) (2024)
- [A Novel Two-Phase Cooperative Co-evolution Framework for Large-Scale Global Optimization with Complex Overlapping](https://mlsanity.com/paper/2503.21797/) (2025)

## Built on by (most cited first, 4 of 4)

- [MetaBox-v2: A Unified Benchmark Platform for Meta-Black-Box Optimization](https://mlsanity.com/paper/2505.17745/) (2025)
- [Detect and Act: Automated Dynamic Optimizer through Meta-Black-Box Optimization](https://mlsanity.com/paper/2601.22542/) (2026)
- [A Learning-Based Cooperative Coevolution Framework for Heterogeneous Large-Scale Global Optimization](https://mlsanity.com/paper/2604.01241/) (2026)
- [Neural Operator-enabled Topology-informed Evolutionary Strategy for PDE-Constrained Optimization](https://mlsanity.com/paper/2607.07682/) (2026)

## Abstract (arXiv)

Recent research in Cooperative Coevolution~(CC) have achieved promising progress in solving large-scale global optimization problems. However, existing CC paradigms have a primary limitation in that they require deep expertise for selecting or designing effective variable decomposition strategies. Inspired by advancements in Meta-Black-Box Optimization, this paper introduces LCC, a pioneering learning-based cooperative coevolution framework that dynamically schedules decomposition strategies during optimization processes. The decomposition strategy selector is parameterized through a neural network, which processes a meticulously crafted set of optimization status features to determine the optimal strategy for each optimization step. The network is trained via the Proximal Policy Optimization method in a reinforcement learning manner across a collection of representative problems, aiming to maximize the expected optimization performance. Extensive experimental results demonstrate that LCC not only offers certain advantages over state-of-the-art baselines in terms of optimization effectiveness and resource consumption, but it also exhibits promising transferability towards unseen problems.