mlsanityData to 14 Oct 2026

Improving Experience Replay through Modeling of Similar Transitions' Sets

arXiv 2111.06907 ↗First posted 12 Nov 2021cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

In this work, we propose and evaluate a new reinforcement learning method, COMPact Experience Replay (COMPER), which uses temporal difference learning with predicted target values based on recurrence over sets of similar transitions, and a new approach for experience replay based on two transitions memories. Our objective is to reduce the required number of experiences to agent training regarding the total accumulated rewarding in the long run. Its relevance to reinforcement learning is related to the small number of observations that it needs to achieve results similar to that obtained by relevant methods in the literature, that generally demand millions of video frames to train an agent on the Atari 2600 games. We report detailed results from five training trials of COMPER for just 100,000 frames and about 25,000 iterations with a small experiences memory on eight challenging games of Arcade Learning Environment (ALE). We also present results for a DQN agent with the same experimental protocol on the same games set as the baseline. To verify the performance of COMPER on approximating a good policy from a smaller number of observations, we also compare its results with that obtained from millions of frames presented on the benchmark of ALE.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 24 of its 33 references that are in the map. First posted on arXiv 12 Nov 2021.

Builds on

24 of 33 references.

References · 24

  1. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  2. 2017Billion-scale similarity search with GPUscited by 2,497
  3. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,258
  4. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,745
  5. 2017Hindsight Experience Replaycited by 1,136
  6. 2015Prioritized Experience Replaycited by 1,099
  7. 2017Rainbow: Combining Improvements in Deep Reinforcement Learningcited by 786
  8. 2015Dueling Network Architectures for Deep Reinforcement Learningcited by 785
  9. 2017A Distributional Perspective on Reinforcement Learningcited by 765
  10. 2019Model-Based Reinforcement Learning for Ataricited by 502
  11. 2015Deep Recurrent Q-Learning for Partially Observable MDPscited by 465
  12. 2017Revisiting the Arcade Learning Environment: Evaluation Protocols and Open Problems for General Agentscited by 269
12 more
  1. 2018Distributed Prioritized Experience Replaycited by 269
  2. 2017Noisy Networks for Explorationcited by 266
  3. 2020Revisiting Fundamentals of Experience Replaycited by 175
  4. 2019When to use parametric models in reinforcement learning?cited by 118
  5. 2017A Deeper Look at Experience Replaycited by 105
  6. 2019Experience Replay Optimizationcited by 48
  7. 2018Remember and Forget for Experience Replaycited by 43
  8. 2015State of the Art Control of Atari Games Using Shallow Reinforcement Learningcited by 23
  9. 2018Organizing Experience: A Deeper Look at Replay Mechanisms for Sample-based Planning in Continuous State Domainscited by 19
  10. 2021Deep Reinforcement Learning with Quantum-inspired Experience Replaycited by 19
  11. 2019Performing Deep Recurrent Double Q-Learning for Atari Gamescited by 5
  12. 2021Stratified Experience Replay: Correcting Multiplicity Bias in Off-Policy Reinforcement Learningcited by 3

Built on by

Not cited yet.