# Effective Reinforcement Learning through Evolutionary Surrogate-Assisted Prescription

arXiv: https://arxiv.org/abs/2002.05368 · HTML: https://mlsanity.com/paper/2002.05368/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Feb 2020.

## Built on by (most cited first, 7 of 7)

- [EVOTER: Evolution of Transparent Explainable Rule-sets](https://mlsanity.com/paper/2204.10438/) (2022)
- [Discovering Effective Policies for Land-Use Planning with Neuroevolution](https://mlsanity.com/paper/2311.12304/) (2023)
- [Surrogate-Assisted Evolutionary Reinforcement Learning Based on Autoencoder and Hyperbolic Neural Network](https://mlsanity.com/paper/2505.19423/) (2025)
- [Optimizing the Design of an Artificial Pancreas to Improve Diabetes Management](https://mlsanity.com/paper/2402.07949/) (2024)
- [Unlocking the Potential of Global Human Expertise](https://mlsanity.com/paper/2411.00156/) (2024)
- [Leveraging Evolutionary Surrogate-Assisted Prescription in Multi-Objective Chlorination Control Systems](https://mlsanity.com/paper/2508.19173/) (2025)
- [Reinforcement Learning with a Bilevel World-Model Architecture for Scan-Order Optimisation in Laser Directed Energy Deposition](https://mlsanity.com/paper/2605.25063/) (2026)

## Abstract (arXiv)

There is now significant historical data available on decision making in organizations, consisting of the decision problem, what decisions were made, and how desirable the outcomes were. Using this data, it is possible to learn a surrogate model, and with that model, evolve a decision strategy that optimizes the outcomes. This paper introduces a general such approach, called Evolutionary Surrogate-Assisted Prescription, or ESP. The surrogate is, for example, a random forest or a neural network trained with gradient descent, and the strategy is a neural network that is evolved to maximize the predictions of the surrogate model. ESP is further extended in this paper to sequential decision-making tasks, which makes it possible to evaluate the framework in reinforcement learning (RL) benchmarks. Because the majority of evaluations are done on the surrogate, ESP is more sample efficient, has lower variance, and lower regret than standard RL approaches. Surprisingly, its solutions are also better because both the surrogate and the strategy network regularize the decision-making behavior. ESP thus forms a promising foundation to decision optimization in real-world problems.