mlsanityData to 7 Oct 2026

Learning Multiple Timescales for Goal-Conditioned Reinforcement Learning

arXiv 2610.00849 ↗First posted 1 Oct 2026cs.AI stat.MLcited by 0

On the map: Offline RL and imitation learning from data → Diffusion and flow policies for offline RL also fits: Offline RL algorithms

Abstract (arXiv)

Existing approaches to offline goal-conditioned reinforcement learning (GCRL) struggle with long-horizon tasks. Discounting shrinks value differences between distant states until they fall below the function approximation error, leaving the agent with no signal for ranking states. Temporal abstraction, which treats k environment steps as a single transition, restores this signal at long range, but no single fixed k suits all state-goal distances: large k preserves value differences across long temporal distances while collapsing distinctions between nearby states, and small k does the reverse. We make this trade-off explicit and introduce Generalized Implicit Temporal Abstraction (GITA), which conditions a single value function on k. GITA trains one policy by aggregating advantage-weighted supervision across multiple k values, so scales assigning larger positive advantages to a state-goal pair contribute more strongly to its update. GITA does not need to choose between local resolution and long-range signal; it retains both without committing to a single k. On OGBench, GITA outperforms a broad range of offline GCRL baselines, raising average success rate across all tasks by 25 percentage points (73% relative improvement) over HIQL. It also improves over the strongest fixed-k method, OTA, by 7 percentage points (14% relative).

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Diffusion and flow policies for offline RL, one of 13 directions in Offline RL and imitation learning from data. Builds on 24 of its 30 references that are in the map. First posted on arXiv 1 Oct 2026.

Builds on

24 of 30 references.

Closest ancestors · 7

Same direction, newest first

  1. 2026Recursive Value Learning for Long-Horizon Offline Goal-Conditioned RL2 Sep
  2. 2026Offline RL with Hierarchical Action Chunking23 Jul
  3. 2026Adaptive Coarse-to-Fine Subgoal Refinement for Long-Horizon Offline Goal-Conditioned Reinforcement Learning27 May
  4. 2026Abstraction for Offline Goal-Conditioned Reinforcement Learning21 May
  5. 2026Chain-of-Goals Hierarchical Policy for Long-Horizon Offline Goal-Conditioned RL3 Feb
  6. 2025Option-aware Temporally Abstracted Value for Offline Goal-Conditioned Reinforcement Learning19 May
  7. 2024OGBench: Benchmarking Offline Goal-Conditioned RL26 Oct

Other references · 17

  1. 2016Layer Normalizationcited by 5,579
  2. 2016Gaussian Error Linear Units (GELUs)cited by 3,431
  3. 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,286
  4. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  5. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,406
  6. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,245
  7. 2017Hindsight Experience Replaycited by 1,096
  8. 2018IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architecturescited by 866
  9. 2019Advantage-Weighted Regression: Simple and Scalable Off-Policy Reinforcement Learningcited by 750
  10. 2022Contrastive Learning as Goal-Conditioned Reinforcement Learningcited by 241
  11. 2023HIQL: Offline Goal-Conditioned RL with Latent States as Actionscited by 133
  12. 2023Optimal Goal-Reaching Reinforcement Learning via Quasimetric Learningcited by 116
5 more
  1. 2025Test-Time Graph Search for Goal-Conditioned Reinforcement Learningcited by 8
  2. 2019Gamma-Nets: Generalizing Value Estimation over Timescalecited by 6
  3. 2026Laplacian Representations for Decision-Time Planningcited by 4
  4. 2026MSPR: Multi-scale Predictive Representations for Goal-conditioned Reinforcement Learningcited by 1
  5. 2026Offline Reinforcement Learning with Universal Horizon Modelscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2023Provably Efficient Offline Goal-Conditioned Reinforcement Learning with General Function Approximation and Single-Policy Concentrability
  2. 2026Beyond Success Rates: Trainability and Extractability for Offline GCRL
  3. 2024Offline Policy Learning via Skill-step Abstraction for Long-horizon Goal-Conditioned Tasks
  4. 2025Flattening Hierarchies with Policy Bootstrapping
  5. 2026Latent Representation Alignment for Offline Goal-Conditioned Reinforcement Learning
  6. 2026PathBridger: Subgoal Bridges for Offline Goal-Conditioned Reinforcement Learning
  7. 2026GTRL: Grounding Divide-and-Conquer Value Learning with Temporal Differences
  8. 2026Improving Offline Goal-Conditioned Reinforcement Learning via Selective Reward Stimulation