mlsanityData to 7 Oct 2026

Event Tables for Efficient Experience Replay

arXiv 2211.00576 ↗First posted 1 Nov 2022cs.LG cs.AIcited by 4

Not on the map. Why?

Abstract (arXiv)

Experience replay (ER) is a crucial component of many deep reinforcement learning (RL) systems. However, uniform sampling from an ER buffer can lead to slow convergence and unstable asymptotic behaviors. This paper introduces Stratified Sampling from Event Tables (SSET), which partitions an ER buffer into Event Tables, each capturing important subsequences of optimal behavior. We prove a theoretical advantage over the traditional monolithic buffer approach and combine SSET with an existing prioritized sampling strategy to further improve learning speed and stability. Empirical results in challenging MiniGrid domains, benchmark RL environments, and a high-fidelity car racing simulator demonstrate the advantages and versatility of SSET over existing ER buffer sampling approaches.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 23 of its 47 references that are in the map. First posted on arXiv 1 Nov 2022.

Builds on

23 of 47 references.

References · 23

  1. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  3. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  4. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,704
  5. 2016OpenAI Gymcited by 1,684
  6. 2017Hindsight Experience Replaycited by 1,096
  7. 2018Experience Replay for Continual Learningcited by 1,074
  8. 2015Prioritized Experience Replaycited by 1,070
  9. 2013An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networkscited by 711
  10. 2021Conflict-Averse Gradient Descent for Multi-task Learningcited by 441
  11. 2018Selective Experience Replay for Lifelong Learningcited by 239
  12. 2016Deep Successor Reinforcement Learningcited by 72
11 more
  1. 2020Super-Human Performance in Gran Turismo Sport Using Deep Reinforcement Learningcited by 60
  2. 2019Experience Replay Optimizationcited by 47
  3. 2021Autonomous Overtaking in Gran Turismo Sport Using Curriculum Reinforcement Learningcited by 41
  4. 2018Sample-Efficient Deep Reinforcement Learning via Episodic Backward Updatecited by 29
  5. 2021Reverb: A Framework For Experience Replaycited by 27
  6. 2018BaRC: Backward Reachability Curriculum for Robotic Reinforcement Learningcited by 21
  7. 2022Topological Experience Replaycited by 17
  8. 2020Periodic Q-Learningcited by 10
  9. 2022A Note on Target Q-learning For Solving Finite MDPs with A Generative Oraclecited by 4
  10. 2020Lucid Dreaming for Experience Replay: Refreshing Past States with the Current Policycited by 3
  11. 2017Experience Replay Using Transition Sequencescited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 1; 2025: 0; 2026: 1.

4 papers.

  1. 2024ETGL-DDPG: A Deep Deterministic Policy Gradient Algorithm for Sparse Reward Continuous Controlcited by 2
  2. 2023Two-Memory Reinforcement Learningcited by 1
  3. 2023Enhanced Generalization through Prioritization and Diversity in Self-Imitation Reinforcement Learning over Procedural Environments with Sparse Rewardscited by 0
  4. 2026Coachable agents for interactive gameplaycited by 0