# Generalized Early Stopping in Evolutionary Direct Policy Search

arXiv: https://arxiv.org/abs/2308.03574 · HTML: https://mlsanity.com/paper/2308.03574/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 55 references that are in the map.
- First posted on arXiv 7 Aug 2023.

## Builds on (most cited first, 12 of 12)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [OpenAI Gym](https://mlsanity.com/paper/1606.01540/) (2016)
- [Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimization](https://mlsanity.com/paper/1603.06560/) (2016)
- [BOHB: Robust and Efficient Hyperparameter Optimization at Scale](https://mlsanity.com/paper/1807.01774/) (2018)
- [ParamILS: An Automatic Algorithm Configuration Framework](https://mlsanity.com/paper/1401.3492/) (2014)
- [Bayesian Surrogate Analysis and Uncertainty Propagation](https://mlsanity.com/paper/2101.04038/) (2021)
- [Morpho-evolution with learning using a controller archive as an inheritance mechanism](https://mlsanity.com/paper/2104.04269/) (2021)
- [A Taxonomy of Constraints in Simulation-Based Optimization](https://mlsanity.com/paper/1505.07881/) (2015)
- [Dynamic Subgoal-based Exploration via Bayesian Optimization](https://mlsanity.com/paper/1910.09143/) (2019)
- [Comparing Two Samples Through Stochastic Dominance: A Graphical Approach](https://mlsanity.com/paper/2203.07889/) (2022)
- [On Solving Cycle Problems with Branch-and-Cut: Extending Shrinking and Exact Subcycle Elimination Separation Algorithms](https://mlsanity.com/paper/2004.14574/) (2020)
- [Extensible Logging and Empirical Attainment Function for IOHexperimenter](https://mlsanity.com/paper/2109.13773/) (2021)

## Built on by (most cited first, 1 of 1)

- [Speeding-up Evolutionary Algorithms to solve Black-Box Optimization Problems](https://mlsanity.com/paper/2309.13349/) (2023)

## Abstract (arXiv)

Lengthy evaluation times are common in many optimization problems such as direct policy search tasks, especially when they involve conducting evaluations in the physical world, e.g. in robotics applications. Often when evaluating solution over a fixed time period it becomes clear that the objective value will not increase with additional computation time (for example when a two wheeled robot continuously spins on the spot). In such cases, it makes sense to stop the evaluation early to save computation time. However, most approaches to stop the evaluation are problem specific and need to be specifically designed for the task at hand. Therefore, we propose an early stopping method for direct policy search. The proposed method only looks at the objective value at each time step and requires no problem specific knowledge. We test the introduced stopping criterion in five direct policy search environments drawn from games, robotics and classic control domains, and show that it can save up to 75% of the computation time. We also compare it with problem specific stopping criteria and show that it performs comparably, while being more generally applicable.