mlsanityData to 7 Oct 2026

Beyond Success Rates: Trainability and Extractability for Offline GCRL

arXiv 2602.05459 ↗First posted 5 Feb 2026cs.LGcited by 0

On the map: Offline RL and imitation learning from data → Diffusion and flow policies for offline RL

Abstract (arXiv)

Offline goal-conditioned reinforcement learning (GCRL) is typically benchmarked by the best tuned success rate of each method. This score measures attainable performance, but it does not reveal how reliably a learned goal-conditioned signal can be extracted into a policy: a method could succeed across many value-learning and extraction settings, or only at a narrow, hard-to-find configuration. We study this gap across four methods, GCIQL, GCIVL, QRL, and CRL, under a shared advantage-weighted regression (AWR) extractor. For each method, we construct trainability landscapes over the optimizer learning rate, which affects value learning and actor optimization, and AWR temperature, which controls how selectively the actor imitates high-advantage transitions. Across AntMaze, Cube, and Scene, we observe distinct regimes: high-scoring methods may be broadly accessible or brittle, while broad relative basins may still sit below low absolute ceilings. To interpret these differences, we pair landscapes with post-hoc diagnostics of future-vs-random goal discrimination and AWR weight concentration. Their relationship to downstream success is task-dependent. On AntMaze, where future goals align with path-like progress, these diagnostics explain landscape regimes. On Cube and Scene, goal ranking and manipulation control decouple: methods can rank goals well while failing downstream, or succeed through action-conditioned advantages despite weak future-vs-random separation. These results show that peak tuned success alone does not establish broadly extractable goal-conditioned behavior. Trainability landscapes expose this gap, while extraction diagnostics offer a lower-cost lens on how learned signals become policies.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Diffusion and flow policies for offline RL, one of 13 directions in Offline RL and imitation learning from data. Builds on 43 of its 58 references that are in the map. First posted on arXiv 5 Feb 2026.

Builds on

43 of 58 references.

Closest ancestors · 9

Same direction, newest first

  1. 2026Chain-of-Goals Hierarchical Policy for Long-Horizon Offline Goal-Conditioned RL3 Feb
  2. 2025Offline Goal-conditioned Reinforcement Learning with Quasimetric Representations24 Sep
  3. 2025Physics-informed Value Learner for Offline Goal-Conditioned Reinforcement Learning8 Sep
  4. 2025Offline Goal-Conditioned Reinforcement Learning with Projective Quasimetric Planning23 Jun
  5. 2025Option-aware Temporally Abstracted Value for Offline Goal-Conditioned Reinforcement Learning19 May
  6. 2024OGBench: Benchmarking Offline Goal-Conditioned RL26 Oct
  7. 2023SMORE: Score Models for Offline Goal-Conditioned Reinforcement Learning3 Nov
  8. 2022How Far I'll Go: Offline Goal-Conditioned Reinforcement Learning via f-Advantage Regression7 Jun
  9. 2022Rethinking Goal-conditioned Supervised Learning and Its Connection to Offline RL9 Feb

Other references · 34

  1. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  2. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,777
  3. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,406
  4. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,245
  5. 2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,227
  6. 2017Hindsight Experience Replaycited by 1,096
  7. 2019Stabilizing Off-Policy Q-Learning via Bootstrapping Error Reductioncited by 833
  8. 2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 781
  9. 2019Advantage-Weighted Regression: Simple and Scalable Off-Policy Reinforcement Learningcited by 750
  10. 2017Deep Reinforcement Learning that Matterscited by 727
  11. 2020AWAC: Accelerating Online Reinforcement Learning with Offline Datasetscited by 647
  12. 2018Visual Reinforcement Learning with Imagined Goalscited by 248
22 more
  1. 2022Contrastive Learning as Goal-Conditioned Reinforcement Learningcited by 241
  2. 2019Planning with Goal-Conditioned Policiescited by 144
  3. 2023HIQL: Offline Goal-Conditioned RL with Latent States as Actionscited by 133
  4. 2021Actionable Models: Unsupervised Offline Reinforcement Learning of Robotic Skillscited by 125
  5. 2023Optimal Goal-Reaching Reinforcement Learning via Quasimetric Learningcited by 116
  6. 2020Hyperparameter Selection for Offline Reinforcement Learningcited by 113
  7. 2020C-Learning: Learning to Achieve Goals via Recursive Classificationcited by 99
  8. 2024Is Value Learning Really the Main Bottleneck in Offline RL?cited by 82
  9. 2022Automated Reinforcement Learning (AutoRL): A Survey and Open Problemscited by 69
  10. 2023Hyperparameters in Reinforcement Learning and How To Tune Themcited by 58
  11. 2025Horizon Reduction Makes RL Scalablecited by 55
  12. 2018Learning Actionable Representations with Goal-Conditioned Policiescited by 53
  13. 2024On the consistency of hyper-parameter selection in value-based deep reinforcement learningcited by 21
  14. 2024Accelerating Goal-Conditioned RL Algorithms and Researchcited by 18
  15. 2024A Method for Evaluating Hyperparameter Sensitivity in Reinforcement Learningcited by 15
  16. 2025Horizon Generalization in Reinforcement Learningcited by 15
  17. 2025Dual Goal Representationscited by 13
  18. 2025Flattening Hierarchies with Policy Bootstrappingcited by 8
  19. 2025Test-Time Graph Search for Goal-Conditioned Reinforcement Learningcited by 8
  20. 2025Multistep Quasimetric Learning for Scalable Goal-conditioned Reinforcement Learningcited by 6
  21. 2024Combining Automated Optimisation of Hyperparameters and Reward Shapecited by 5
  22. 2025DeepCAVE: A Visualization and Analysis Tool for Automated Machine Learningcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2023Provably Efficient Offline Goal-Conditioned Reinforcement Learning with General Function Approximation and Single-Policy Concentrability
  2. 2026Latent Representation Alignment for Offline Goal-Conditioned Reinforcement Learning
  3. 2026Learning Multiple Timescales for Goal-Conditioned Reinforcement Learning
  4. 2026PathBridger: Subgoal Bridges for Offline Goal-Conditioned Reinforcement Learning
  5. 2023What is Essential for Unseen Goal Generalization of Offline Goal-conditioned RL?
  6. 2024Offline Policy Learning via Skill-step Abstraction for Long-horizon Goal-Conditioned Tasks
  7. 2026Efficient Hierarchical Implicit Flow Q-learning for Offline Goal-conditioned Reinforcement Learning
  8. 2026Abstraction for Offline Goal-Conditioned Reinforcement Learning