# Combining Experience Replay with Exploration by Random Network Distillation

arXiv: https://arxiv.org/abs/1905.07579 · HTML: https://mlsanity.com/paper/1905.07579/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 May 2019.

## Built on by (most cited first, 5 of 5)

- [Generalized Proximal Policy Optimization with Sample Reuse](https://mlsanity.com/paper/2111.00072/) (2021)
- [Towards Improving Exploration in Self-Imitation Learning using Intrinsic Motivation](https://mlsanity.com/paper/2211.16838/) (2022)
- [Sample Dropout: A Simple yet Effective Variance Reduction Technique in Deep Policy Optimization](https://mlsanity.com/paper/2302.02299/) (2023)
- [Explanation-Aware Experience Replay in Rule-Dense Environments](https://mlsanity.com/paper/2109.14711/) (2021)
- [Random Network Distillation Based Deep Reinforcement Learning for AGV Path Planning](https://mlsanity.com/paper/2404.12594/) (2024)

## Abstract (arXiv)

Our work is a simple extension of the paper "Exploration by Random Network Distillation". More in detail, we show how to efficiently combine Intrinsic Rewards with Experience Replay in order to achieve more efficient and robust exploration (with respect to PPO/RND) and consequently better results in terms of agent performances and sample efficiency. We are able to do it by using a new technique named Prioritized Oversampled Experience Replay (POER), that has been built upon the definition of what is the important experience useful to replay. Finally, we evaluate our technique on the famous Atari game Montezuma's Revenge and some other hard exploration Atari games.