# Optimal Use of Experience in First Person Shooter Environments

arXiv: https://arxiv.org/abs/1906.09734 · HTML: https://mlsanity.com/paper/1906.09734/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Jun 2019.

## Built on by (most cited first, 1 of 1)

- [DNA: Proximal Policy Optimization with a Dual Network Architecture](https://mlsanity.com/paper/2206.10027/) (2022)

## Abstract (arXiv)

Although reinforcement learning has made great strides recently, a continuing limitation is that it requires an extremely high number of interactions with the environment. In this paper, we explore the effectiveness of reusing experience from the experience replay buffer in the Deep Q-Learning algorithm. We test the effectiveness of applying learning update steps multiple times per environmental step in the VizDoom environment and show first, this requires a change in the learning rate, and second that it does not improve the performance of the agent. Furthermore, we show that updating less frequently is effective up to a ratio of 4:1, after which performance degrades significantly. These results quantitatively confirm the widespread practice of performing learning updates every 4th environmental step.