# Surrogate Learning in Meta-Black-Box Optimization: A Preliminary Study

arXiv: https://arxiv.org/abs/2503.18060 · HTML: https://mlsanity.com/paper/2503.18060/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Placed in RL for games and combinatorial optimization, one of 15 directions in Deep reinforcement learning methods.
- Builds on 15 of its 44 references that are in the map.
- First posted on arXiv 23 Mar 2025.

## Placed in

- Deep reinforcement learning methods / [RL for games and combinatorial optimization](https://mlsanity.com/deeprl/rl-for-games-and-combinatorial-optimization/)

## Builds on (most cited first, 15 of 15)

- [The CMA Evolution Strategy: A Tutorial](https://mlsanity.com/paper/1604.00772/) (2016)
- [Deep UQ: Learning deep neural network surrogate models for high dimensional uncertainty quantification](https://mlsanity.com/paper/1802.00850/) (2018)
- [KAN or MLP: A Fairer Comparison](https://mlsanity.com/paper/2407.16674/) (2024)
- [Discovering Evolution Strategies via Meta-Black-Box Optimization](https://mlsanity.com/paper/2211.11260/) (2022)
- [Deep Reinforcement Learning Based Parameter Control in Differential Evolution](https://mlsanity.com/paper/1905.08006/) (2019)
- [MetaBox: A Benchmark Platform for Meta-Black-Box Optimization with Reinforcement Learning](https://mlsanity.com/paper/2310.08252/) (2023)
- [Discovering Attention-Based Genetic Algorithms via Meta-Black-Box Optimization](https://mlsanity.com/paper/2304.03995/) (2023)
- [Deep Reinforcement Learning for Dynamic Algorithm Selection: A Proof-of-Principle Study on Differential Evolution](https://mlsanity.com/paper/2403.02131/) (2024)
- [Predictive Modeling of Flexible EHD Pumps using Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2405.07488/) (2024)
- [Symbol: Generating Flexible Black-Box Optimizers through Symbolic Equation Learning](https://mlsanity.com/paper/2402.02355/) (2024)
- [Smooth Kolmogorov Arnold networks enabling structural knowledge representation](https://mlsanity.com/paper/2405.11318/) (2024)
- [Auto-configuring Exploration-Exploitation Tradeoff in Evolutionary Computation via Deep Reinforcement Learning](https://mlsanity.com/paper/2404.08239/) (2024)
- [B2Opt: Learning to Optimize Black-box Optimization with Little Budget](https://mlsanity.com/paper/2304.11787/) (2023)
- [Offline Model-Based Optimization by Learning to Rank](https://mlsanity.com/paper/2410.11502/) (2024)
- [A First Look at Kolmogorov-Arnold Networks in Surrogate-assisted Evolutionary Algorithms](https://mlsanity.com/paper/2405.16494/) (2024)

## Built on by (most cited first, 7 of 7)

- [MetaBox-v2: A Unified Benchmark Platform for Meta-Black-Box Optimization](https://mlsanity.com/paper/2505.17745/) (2025)
- [DesignX: Human-Competitive Algorithm Designer for Black-Box Optimization](https://mlsanity.com/paper/2505.17866/) (2025)
- [Advancing CMA-ES with Learning-Based Cooperative Coevolution for Scalable Optimization](https://mlsanity.com/paper/2504.17578/) (2025)
- [Learning Where, What and How to Transfer: A Multi-Role Reinforcement Learning Approach for Evolutionary Multitasking](https://mlsanity.com/paper/2511.15199/) (2025)
- [Deep Reinforcement Learning for Dynamic Algorithm Configuration: A Case Study on Optimizing OneMax with the (1+(λ,λ))-GA](https://mlsanity.com/paper/2512.03805/) (2025)
- [Detect and Act: Automated Dynamic Optimizer through Meta-Black-Box Optimization](https://mlsanity.com/paper/2601.22542/) (2026)
- [Surrogate Ensemble in Expensive Multi-Objective Optimization via Deep Q-Learning](https://mlsanity.com/paper/2602.00540/) (2026)

## Abstract (arXiv)

Recent Meta-Black-Box Optimization (MetaBBO) approaches have shown possibility of enhancing the optimization performance through learning meta-level policies to dynamically configure low-level optimizers. However, existing MetaBBO approaches potentially consume massive function evaluations to train their meta-level policies. Inspired by the recent trend of using surrogate models for cost-friendly evaluation of expensive optimization problems, in this paper, we propose a novel MetaBBO framework which combines surrogate learning process and reinforcement learning-aided Differential Evolution algorithm, namely Surr-RLDE, to address the intensive function evaluation in MetaBBO. Surr-RLDE comprises two learning stages: surrogate learning and policy learning. In surrogate learning, we train a Kolmogorov-Arnold Networks (KAN) with a novel relative-order-aware loss to accurately approximate the objective functions of the problem instances used for subsequent policy learning. In policy learning, we employ reinforcement learning (RL) to dynamically configure the mutation operator in DE. The learned surrogate model is integrated into the training of the RL-based policy to substitute for the original objective function, which effectively reduces consumed evaluations during policy learning. Extensive benchmark results demonstrate that Surr-RLDE not only shows competitive performance to recent baselines, but also shows compelling generalization for higher-dimensional problems. Further ablation studies underscore the effectiveness of each technical components in Surr-RLDE. We open-source Surr-RLDE at https://github.com/GMC-DRL/Surr-RLDE.