mlsanityData to 7 Oct 2026

Prioritized offline Goal-swapping Experience Replay

arXiv 2302.07741 ↗First posted 15 Feb 2023cs.LG cs.AIcited by 0

On the map: Offline RL and imitation learning from data → Diffusion and flow policies for offline RL

Abstract (arXiv)

In goal-conditioned offline reinforcement learning, an agent learns from previously collected data to go to an arbitrary goal. Since the offline data only contains a finite number of trajectories, a main challenge is how to generate more data. Goal-swapping generates additional data by switching trajectory goals but while doing so produces a large number of invalid trajectories. To address this issue, we propose prioritized goal-swapping experience replay (PGSER). PGSER uses a pre-trained Q function to assign higher priority weights to goal swapped transitions that allow reaching the goal. In experiments, PGSER significantly improves over baselines in a wide range of benchmark tasks, including challenging previously unsuccessful dexterous in-hand manipulation tasks.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Diffusion and flow policies for offline RL, one of 13 directions in Offline RL and imitation learning from data. Builds on 18 of its 24 references that are in the map. First posted on arXiv 15 Feb 2023.

Builds on

18 of 24 references.

Closest ancestors · 1

Same direction, newest first

  1. 2022Rethinking Goal-conditioned Supervised Learning and Its Connection to Offline RL9 Feb

Other references · 17

  1. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  2. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
  3. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  4. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,777
  5. 2017Hindsight Experience Replaycited by 1,096
  6. 2015Prioritized Experience Replaycited by 1,070
  7. 2021A Minimalist Approach to Offline Reinforcement Learningcited by 882
  8. 2021Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learningcited by 337
  9. 2018Multi-Goal Reinforcement Learning: Challenging Robotics Environments and Request for Researchcited by 256
  10. 2022Contrastive Learning as Goal-Conditioned Reinforcement Learningcited by 241
  11. 2022A Survey on Offline Reinforcement Learning: Taxonomy, Review, and Open Problemscited by 228
  12. 2022Goal-Conditioned Reinforcement Learning: Problems and Solutionscited by 145
5 more
  1. 2021Goal-Conditioned Reinforcement Learning with Imagined Subgoalscited by 129
  2. 2021Actionable Models: Unsupervised Offline Reinforcement Learning of Robotic Skillscited by 125
  3. 2021On Improving Model-Free Algorithms for Decentralized Multi-Agent Reinforcement Learningcited by 35
  4. 2022Metric Residual Networks for Sample Efficient Goal-Conditioned Reinforcement Learningcited by 12
  5. 2019Improving Deep Reinforcement Learning in Minecraft with Action Advicecited by 10

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2023Swapped goal-conditioned offline reinforcement learning
  2. 2025Test-time Offline Reinforcement Learning on Goal-related Experience
  3. 2023Decoupled Prioritized Resampling for Offline RL
  4. 2022How Far I'll Go: Offline Goal-Conditioned Reinforcement Learning via f-Advantage Regression
  5. 2023Prioritized Trajectory Replay: A Replay Memory for Data-driven Reinforcement Learning
  6. 2024Stitching Sub-Trajectories with Conditional Diffusion Model for Goal-Conditioned Offline RL
  7. 2023OER: Offline Experience Replay for Continual Offline Reinforcement Learning
  8. 2025Reachability Weighted Offline Goal-conditioned Resampling