mlsanityData to 14 Oct 2026

Prioritized Sequence Experience Replay

arXiv 1905.12726 ↗First posted 25 May 2019cs.LG cs.AI stat.MLcited by 20

Not on the map. Why?

Abstract (arXiv)

Experience replay is widely used in deep reinforcement learning algorithms and allows agents to remember and learn from experiences from the past. In an effort to learn more efficiently, researchers proposed prioritized experience replay (PER) which samples important transitions more frequently. In this paper, we propose Prioritized Sequence Experience Replay (PSER) a framework for prioritizing sequences of experience in an attempt to both learn more efficiently and to obtain better performance. We compare the performance of PER and PSER sampling techniques in a tabular Q-learning environment and in DQN on the Atari 2600 benchmark. We prove theoretically that PSER is guaranteed to converge faster than PER and empirically show PSER substantially improves upon PER.

Key facts. Cited by 20 papers in the map (data through 14 Oct 2026). First posted on arXiv 25 May 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 6; 2022: 5; 2023: 4; 2024: 0; 2025: 4; 2026: 1.

20 papers.

Sort:

By area:

  1. 2021Regret Minimization Experience Replay in Off-Policy Reinforcement Learningcited by 25
  2. 2022Actor Prioritized Experience Replaycited by 20
  3. 2023MAC-PO: Multi-Agent Experience Replay via Collective Priority Optimizationcited by 20
  4. 2022Live in the Moment: Learning Dynamics Model Adapted to Evolving Policycited by 17
  5. 2021Improved Soft Actor-Critic: Mixing Prioritized Off-Policy Samples with On-Policy Experiencecited by 8
  6. 2021Revisiting Prioritized Experience Replay: A Value Perspectivecited by 7
  7. 2022From Psychological Curiosity to Artificial Curiosity: Curiosity-Driven Learning in Artificial Intelligence Taskscited by 6
  8. 2023Decoupled Prioritized Resampling for Offline RLcited by 6
  9. 2021State-Aware Variational Thompson Sampling for Deep Q-Networkscited by 5
  10. 2025Active Advantage-Aligned Online Reinforcement Learning with Offline Datacited by 5
  11. 2022Asynchronous Curriculum Experience Replay: A Deep Reinforcement Learning Approach for UAV Autonomous Motion Control in Unknown Dynamic Environmentscited by 4
  12. 2025Reliability-Adjusted Prioritized Experience Replaycited by 3
8 more
  1. 2025Sample Efficient Experience Replay in Non-stationary Environmentscited by 3
  2. 2023Hindsight States: Blending Sim and Real Task Elements for Efficient Reinforcement Learningcited by 2
  3. 2021Model-Augmented Q-learningcited by 1
  4. 2023Directly Attention Loss Adjusted Prioritized Experience Replaycited by 1
  5. 2021Improving Experience Replay with Successor Representationcited by 0
  6. 2022MEET: A Monte Carlo Exploration-Exploitation Trade-off for Buffer Samplingcited by 0
  7. 2025EVaDE : Event-Based Variational Thompson Sampling for Model-Based Reinforcement Learningcited by 0
  8. 2026Uncertainty-Driven Replay Memory for Reinforcement Learningcited by 0