# Mixing Human Demonstrations with Self-Exploration in Experience Replay for Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/2107.06840 · HTML: https://mlsanity.com/paper/2107.06840/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 7 of its 11 references that are in the map.
- First posted on arXiv 14 Jul 2021.

## Builds on (most cited first, 7 of 7)

- [Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments](https://mlsanity.com/paper/1706.02275/) (2017)
- [Overcoming Exploration in Reinforcement Learning with Demonstrations](https://mlsanity.com/paper/1709.10089/) (2017)
- [Selective Experience Replay for Lifelong Learning](https://mlsanity.com/paper/1802.10269/) (2018)
- [Reinforcement Learning from Imperfect Demonstrations](https://mlsanity.com/paper/1802.05313/) (2018)
- [Learning Arbitrary-Goal Fabric Folding with One Hour of Real Robot Experience](https://mlsanity.com/paper/2010.03209/) (2020)
- [Decentralized Multi-Agent Pursuit using Deep Reinforcement Learning](https://mlsanity.com/paper/2010.08193/) (2020)
- [Passing Through Narrow Gaps with Deep Reinforcement Learning](https://mlsanity.com/paper/2103.03991/) (2021)

## Abstract (arXiv)

We investigate the effect of using human demonstration data in the replay buffer for Deep Reinforcement Learning. We use a policy gradient method with a modified experience replay buffer where a human demonstration experience is sampled with a given probability. We analyze different ratios of using demonstration data in a task where an agent attempts to reach a goal while avoiding obstacles. Our results suggest that while the agents trained by pure self-exploration and pure demonstration had similar success rates, the pure demonstration model converged faster to solutions with less number of steps.