mlsanityData to 14 Oct 2026

SegBench-GC: Testing Segmentation Invariance in Multi-Step Offline Goal-Conditioned Reinforcement Learning

arXiv 2608.27678 ↗First posted 27 Aug 2026cs.LGcited by 0

On the map: Offline RL and imitation learning from data → Diffusion and flow policies for offline RL

Abstract (arXiv)

Offline goal-conditioned reinforcement learning (GCRL) often uses trajectory structure for future-goal sampling and multi-step targets, yet logged trajectories may be partitioned for administrative reasons that do not correspond to termination. We introduce SegBench-GC, a controlled stress test of segmentation invariance that holds transitions, source trajectories, goal sampling, optimization settings, and evaluation fixed while varying only artificial backup boundaries and whether those boundaries retain continuation value. Continuation-valid targets (CVT) provide the segmentation-consistent control: reward accumulation stops at an artificial cut, but the target bootstraps from its stored successor. In a matched-count PointMaze study with 35,000 artificial cuts, three segmentation realizations, and three optimization seeds, final 50-episode-per-task success is 50.5% uncut, 39.1% with CVT, and 19.1% when the same cuts are treated as absorbing; across segmentation realizations, naive mean success ranges from 4.8% to 31.9%. An independent published n-step baseline (n=25) from the Decoupled Q-Chunking codebase shows the same failure on Puzzle-4x5: 47.2% uncut, 58.5% CVT, and 0.27% naive across three optimization seeds. A target-level diagnostic verifies the analytic target difference to numerical precision, and learned-critic diagnostics show a large optimistic shift under naive handling while CVT remains approximately aligned with the uncut critic. CVT applies standard continuation bootstrapping rather than a new Bellman rule; the contribution is the controlled benchmark, failure isolation, and cross-learner evidence that administrative segmentation can materially change multi-step offline GCRL.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Diffusion and flow policies for offline RL, one of 13 directions in Offline RL and imitation learning from data. Builds on 10 of its 10 references that are in the map. First posted on arXiv 27 Aug 2026.

Builds on

10 of 10 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025Option-aware Temporally Abstracted Value for Offline Goal-Conditioned Reinforcement Learning19 May
  2. 2024OGBench: Benchmarking Offline Goal-Conditioned RL26 Oct

Other references · 8

  1. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,809
  2. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,427
  3. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,293
  4. 2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 807
  5. 2017Time Limits in Reinforcement Learningcited by 79
  6. 2025Horizon Reduction Makes RL Scalablecited by 57
  7. 2025Decoupled Q-Chunkingcited by 20
  8. 2025Transitive RL: Value Learning via Divide and Conquercited by 12

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Test-Time Graph Search for Goal-Conditioned Reinforcement Learning
  2. 2025Test-time Offline Reinforcement Learning on Goal-related Experience
  3. 2026Learning Multiple Timescales for Goal-Conditioned Reinforcement Learning
  4. 2026Beyond Success Rates: Trainability and Extractability for Offline GCRL
  5. 2026Do Better Goal Representations Improve Goal-Conditioned Reinforcement Learning?
  6. 2026PathBridger: Subgoal Bridges for Offline Goal-Conditioned Reinforcement Learning
  7. 2024Offline Policy Learning via Skill-step Abstraction for Long-horizon Goal-Conditioned Tasks
  8. 2025Flattening Hierarchies with Policy Bootstrapping