# SEERL: Sample Efficient Ensemble Reinforcement Learning

arXiv: https://arxiv.org/abs/2001.05209 · HTML: https://mlsanity.com/paper/2001.05209/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Jan 2020.

## Built on by (most cited first, 11 of 11)

- [Deep Reinforcement Learning at the Edge of the Statistical Precipice](https://mlsanity.com/paper/2108.13264/) (2021)
- [Towards Applicable Reinforcement Learning: Improving the Generalization and Sample Efficiency with Policy Ensemble](https://mlsanity.com/paper/2205.09284/) (2022)
- [Ensemble Reinforcement Learning: A Survey](https://mlsanity.com/paper/2303.02618/) (2023)
- [MEPG: A Minimalist Ensemble Policy Gradient Framework for Deep Reinforcement Learning](https://mlsanity.com/paper/2109.10552/) (2021)
- [ARLO: A Framework for Automated Reinforcement Learning](https://mlsanity.com/paper/2205.10416/) (2022)
- [DEFT: Diverse Ensembles for Fast Transfer in Reinforcement Learning](https://mlsanity.com/paper/2209.12412/) (2022)
- [Balancing Value Underestimation and Overestimation with Realistic Actor-Critic](https://mlsanity.com/paper/2110.09712/) (2021)
- [Dataset Clustering for Improved Offline Policy Learning](https://mlsanity.com/paper/2402.09550/) (2024)
- [Scaling DRL for Decision Making: A Survey on Data, Network, and Training Budget Strategies](https://mlsanity.com/paper/2508.03194/) (2025)
- [Visual processing in context of reinforcement learning](https://mlsanity.com/paper/2208.12525/) (2022)
- [Group-Agent Reinforcement Learning with Heterogeneous Agents](https://mlsanity.com/paper/2501.11818/) (2025)

## Abstract (arXiv)

Ensemble learning is a very prevalent method employed in machine learning. The relative success of ensemble methods is attributed to their ability to tackle a wide range of instances and complex problems that require different low-level approaches. However, ensemble methods are relatively less popular in reinforcement learning owing to the high sample complexity and computational expense involved in obtaining a diverse ensemble. We present a novel training and model selection framework for model-free reinforcement algorithms that use ensembles of policies obtained from a single training run. These policies are diverse in nature and are learned through directed perturbation of the model parameters at regular intervals. We show that learning and selecting an adequately diverse set of policies is required for a good ensemble while extreme diversity can prove detrimental to overall performance. Selection of an adequately diverse set of policies is done through our novel policy selection framework. We evaluate our approach on challenging discrete and continuous control tasks and also discuss various ensembling strategies. Our framework is substantially sample efficient, computationally inexpensive and is seen to outperform state-of-the-art (SOTA) scores in Atari 2600 and Mujoco.