# Importance mixing: Improving sample reuse in evolutionary policy search methods

arXiv: https://arxiv.org/abs/1808.05832 · HTML: https://mlsanity.com/paper/1808.05832/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Aug 2018.

## Built on by (most cited first, 4 of 4)

- [Deep Reinforcement Learning Versus Evolution Strategies: A Comparative Survey](https://mlsanity.com/paper/2110.01411/) (2021)
- [Derivative-Free Reinforcement Learning: A Review](https://mlsanity.com/paper/2102.05710/) (2021)
- [Evolving Multi-Objective Neural Network Controllers for Robot Swarms](https://mlsanity.com/paper/2307.14237/) (2023)
- [A Scalable Finite Difference Method for Deep Reinforcement Learning](https://mlsanity.com/paper/2210.07487/) (2022)

## Abstract (arXiv)

Deep neuroevolution, that is evolutionary policy search methods based on deep neural networks, have recently emerged as a competitor to deep reinforcement learning algorithms due to their better parallelization capabilities. However, these methods still suffer from a far worse sample efficiency. In this paper we investigate whether a mechanism known as "importance mixing" can significantly improve their sample efficiency. We provide a didactic presentation of importance mixing and we explain how it can be extended to reuse more samples. Then, from an empirical comparison based on a simple benchmark, we show that, though it actually provides better sample efficiency, it is still far from the sample efficiency of deep reinforcement learning, though it is more stable.