mlsanityData to 7 Oct 2026

Reachability Weighted Offline Goal-conditioned Resampling

arXiv 2506.02577 ↗First posted 3 Jun 2025cs.LGcited by 0

On the map: Offline RL and imitation learning from data → Diffusion and flow policies for offline RL

Abstract (arXiv)

Offline goal-conditioned reinforcement learning (RL) relies on fixed datasets where many potential goals share the same state and action spaces. However, these potential goals are not explicitly represented in the collected trajectories. To learn a generalizable goal-conditioned policy, it is common to sample goals and state-action pairs uniformly using dynamic programming methods such as Q-learning. Uniform sampling, however, requires an intractably large dataset to cover all possible combinations and creates many unreachable state-goal-action pairs that degrade policy performance. Our key insight is that sampling should favor transitions that enable goal achievement. To this end, we propose Reachability Weighted Sampling (RWS). RWS uses a reachability classifier trained via positive-unlabeled (PU) learning on goal-conditioned state-action values. The classifier maps these values to a reachability score, which is then used as a sampling priority. RWS is a plug-and-play module that integrates seamlessly with standard offline RL algorithms. Experiments on six complex simulated robotic manipulation tasks, including those with a robot arm and a dexterous hand, show that RWS significantly improves performance. In one notable case, performance on the HandBlock-Z task improved by nearly 50 percent relative to the baseline. These results indicate the effectiveness of reachability-weighted sampling.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Diffusion and flow policies for offline RL, one of 13 directions in Offline RL and imitation learning from data. Builds on 30 of its 37 references that are in the map. First posted on arXiv 3 Jun 2025.

Builds on

30 of 37 references.

Closest ancestors · 5

Same direction, newest first

  1. 2024OGBench: Benchmarking Offline Goal-Conditioned RL26 Oct
  2. 2023GOPlan: Goal-conditioned Offline Reinforcement Learning by Planning with Learned Models30 Oct
  3. 2022How Far I'll Go: Offline Goal-Conditioned Reinforcement Learning via f-Advantage Regression7 Jun
  4. 2022Hierarchical Planning Through Goal-Conditioned Offline Reinforcement Learning24 May
  5. 2022Rethinking Goal-conditioned Supervised Learning and Its Connection to Offline RL9 Feb

Other references · 25

  1. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  2. 2017Hindsight Experience Replaycited by 1,096
  3. 2019Advantage-Weighted Regression: Simple and Scalable Off-Policy Reinforcement Learningcited by 750
  4. 2020Critic Regularized Regressioncited by 269
  5. 2018Multi-Goal Reinforcement Learning: Challenging Robotics Environments and Request for Researchcited by 256
  6. 2019DualDICE: Behavior-Agnostic Estimation of Discounted Stationary Distribution Correctionscited by 254
  7. 2022Contrastive Learning as Goal-Conditioned Reinforcement Learningcited by 241
  8. 2017Positive-Unlabeled Learning with Non-Negative Risk Estimatorcited by 217
  9. 2019AlgaeDICE: Policy Gradient from Arbitrary Experiencecited by 187
  10. 2022Goal-Conditioned Reinforcement Learning: Problems and Solutionscited by 145
  11. 2021Actionable Models: Unsupervised Offline Reinforcement Learning of Robotic Skillscited by 125
  12. 2022Offline Reinforcement Learning with Realizability and Single-policy Concentrabilitycited by 122
13 more
  1. 2021OptiDICE: Offline Policy Optimization via Stationary Distribution Correction Estimationcited by 121
  2. 2023Optimal Goal-Reaching Reinforcement Learning via Quasimetric Learningcited by 116
  3. 2020C-Learning: Learning to Achieve Goals via Recursive Classificationcited by 99
  4. 2023Reinforcement Learning from Passive Data via Latent Intentionscited by 64
  5. 2023Contrastive Difference Predictive Codingcited by 35
  6. 2022Optimal Conservative Offline RL with General Function Approximation via Augmented Lagrangiancited by 29
  7. 2023Learning Goal-Conditioned Policies Offline with Self-Supervised Reward Shapingcited by 25
  8. 2023Beyond Uniform Sampling: Offline Reinforcement Learning with Imbalanced Datasetscited by 24
  9. 2023Distance Weighted Supervised Learning for Offline Interaction Datacited by 18
  10. 2023f-Policy Gradients: A General Framework for Goal Conditioned RL using f-Divergencescited by 14
  11. 2023GO-DICE: Goal-Conditioned Option-Aware Offline Imitation Learning via Stationary Distribution Correction Estimationcited by 6
  12. 2024Goal-Conditioned Data Augmentation for Offline Reinforcement Learningcited by 4
  13. 2024Q-WSL: Optimizing Goal-Conditioned RL with Weighted Supervised Learning via Dynamic Programmingcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2023Harnessing Mixed Offline Reinforcement Learning Datasets via Trajectory Weighting
  2. 2023Goal-conditioned Offline Reinforcement Learning through State Space Partitioning
  3. 2024MGDA: Model-based Goal Data Augmentation for Offline Goal-conditioned Weighted Supervised Learning
  4. 2025Prompting Decision Transformers for Zero-Shot Reach-Avoid Policies
  5. 2024Goal-Conditioned Decision Transformer for Multi-Goal Offline Reinforcement Learning
  6. 2024Stitching Sub-Trajectories with Conditional Diffusion Model for Goal-Conditioned Offline RL
  7. 2024Offline Safe Reinforcement Learning Using Trajectory Classification
  8. 2026Improving Zero-Shot Offline RL via Behavioral Task Sampling