# Sample-Efficient and Safe Deep Reinforcement Learning via Reset Deep Ensemble Agents

arXiv: https://arxiv.org/abs/2310.20287 · HTML: https://mlsanity.com/paper/2310.20287/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 17 references that are in the map.
- First posted on arXiv 31 Oct 2023.

## Builds on (most cited first, 10 of 10)

- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Soft Actor-Critic Algorithms and Applications](https://mlsanity.com/paper/1812.05905/) (2018)
- [Constrained Policy Optimization](https://mlsanity.com/paper/1705.10528/) (2017)
- [DeepMind Control Suite](https://mlsanity.com/paper/1801.00690/) (2018)
- [Deep Reinforcement Learning at the Edge of the Statistical Precipice](https://mlsanity.com/paper/2108.13264/) (2021)
- [Deep Exploration via Bootstrapped DQN](https://mlsanity.com/paper/1602.04621/) (2016)
- [SUNRISE: A Simple Unified Framework for Ensemble Learning in Deep Reinforcement Learning](https://mlsanity.com/paper/2007.04938/) (2020)
- [Averaged-DQN: Variance Reduction and Stabilization for Deep Reinforcement Learning](https://mlsanity.com/paper/1611.01929/) (2016)
- [Policy Expansion for Bridging Offline-to-Online Reinforcement Learning](https://mlsanity.com/paper/2302.00935/) (2023)
- [Quantile Constrained Reinforcement Learning: A Reinforcement Learning Framework Constraining Outage Probability](https://mlsanity.com/paper/2211.15034/) (2022)

## Built on by (most cited first, 12 of 12)

- [Hyperspherical Normalization for Scalable Deep Reinforcement Learning](https://mlsanity.com/paper/2502.15280/) (2025)
- [Neuroplastic Expansion in Deep Reinforcement Learning](https://mlsanity.com/paper/2410.07994/) (2024)
- [Dissecting Deep RL with High Update Ratios: Combatting Value Divergence](https://mlsanity.com/paper/2403.05996/) (2024)
- [Penalizing Infeasible Actions and Reward Scaling in Reinforcement Learning with Offline Data](https://mlsanity.com/paper/2507.08761/) (2025)
- [Cooperative Backdoor Attack in Decentralized Reinforcement Learning with Theoretical Guarantee](https://mlsanity.com/paper/2405.15245/) (2024)
- [Self-Improving Skill Learning for Robust Skill-based Meta-Reinforcement Learning](https://mlsanity.com/paper/2502.03752/) (2025)
- [Mitigating Suboptimality of Deterministic Policy Gradients in Complex Q-functions](https://mlsanity.com/paper/2410.11833/) (2024)
- [A Comprehensive Survey on Physical Risk Control in the Era of Foundation Model-enabled Robotics](https://mlsanity.com/paper/2505.12583/) (2025)
- [AltNet: Addressing the Plasticity-Stability Dilemma in Reinforcement Learning](https://mlsanity.com/paper/2512.01034/) (2025)
- [GenSafe: A Generalizable Safety Enhancer for Safe Reinforcement Learning Algorithms Based on Reduced Order Markov Decision Process Model](https://mlsanity.com/paper/2406.03912/) (2024)
- [GB-DQN: Gradient Boosted DQN Models for Non-stationary Reinforcement Learning](https://mlsanity.com/paper/2512.17034/) (2025)
- [SingularClip: Preventing Spectral Collapse to Maintain Plasticity in Continual and Reinforcement Learning](https://mlsanity.com/paper/2608.18319/) (2026)

## Abstract (arXiv)

Deep reinforcement learning (RL) has achieved remarkable success in solving complex tasks through its integration with deep neural networks (DNNs) as function approximators. However, the reliance on DNNs has introduced a new challenge called primacy bias, whereby these function approximators tend to prioritize early experiences, leading to overfitting. To mitigate this primacy bias, a reset method has been proposed, which performs periodic resets of a portion or the entirety of a deep RL agent while preserving the replay buffer. However, the use of the reset method can result in performance collapses after executing the reset, which can be detrimental from the perspective of safe RL and regret minimization. In this paper, we propose a new reset-based method that leverages deep ensemble learning to address the limitations of the vanilla reset method and enhance sample efficiency. The proposed method is evaluated through various experiments including those in the domain of safe RL. Numerical results show its effectiveness in high sample efficiency and safety considerations.