mlsanityData to 14 Oct 2026

Hitting Time Isomorphism for Multi-Stage Planning with Foundation Policies

arXiv 2605.06470 ↗First posted 7 May 2026cs.LGcited by 1

On the map: Offline RL and imitation learning from data → Offline RL algorithms

also in 1 more
Abstract (arXiv)

We present a new operator-theoretic representation learning framework for offline reinforcement learning that recovers the directed temporal geometry of a controlled Markov process from hitting time observations. While prior art often produces symmetric distances or fails to satisfy the triangle inequality, our framework learns a Hilbert-space displacement geometry where expected hitting times are realized as linear functionals of latent displacements. We prove that this representation exists under latent linear closure and is uniquely identifiable up to a bounded linear isomorphism. For finite-dimensional implementations, we show that global hitting-time error is bounded by one-step transition error amplified by the environment's transient spectral radius. Furthermore, we provide finite-sample guarantees accounting for approximation, statistical complexity, and trajectory-label mismatch. Derived from this theory, we curate Isomorphic Embedding Learning (IEL) as a new goal-agnostic foundation policy learning algorithm that anchors a HILP-style consistency objective with explicit hitting-time regression to ensure that the learned geometry reflects actual decision-time progress. This asymmetric and compositional structure enables robust graph-based multi-stage planning for long-horizon navigation. Our experiments demonstrate that IEL improves the state of the art of learning foundation policy policies from offline maze locomotion data. Our code can be found on https://github.com/MagnusBoock/IEL

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Offline RL algorithms, one of 13 directions in Offline RL and imitation learning from data. Builds on 23 of its 32 references that are in the map. First posted on arXiv 7 May 2026.

Builds on

23 of 32 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025Graph-Assisted Stitching for Offline Hierarchical Reinforcement Learning9 Jun
  2. 2021Offline Reinforcement Learning with Implicit Q-Learning12 Oct
  3. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learning15 Apr

Other references · 20

  1. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
  2. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,887
  3. 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,407
  4. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,061
  5. 2022A Generalist Agentcited by 808
  6. 2021Modern Koopman Theory for Dynamical Systemscited by 276
  7. 2022Contrastive Learning as Goal-Conditioned Reinforcement Learningcited by 253
  8. 2020Learning Near Optimal Policies with Low Inherent Bellman Errorcited by 181
  9. 2023HIQL: Offline Goal-Conditioned RL with Latent States as Actionscited by 141
  10. 2021URLB: Unsupervised Reinforcement Learning Benchmarkcited by 139
  11. 2023Optimal Goal-Reaching Reinforcement Learning via Quasimetric Learningcited by 121
  12. 2021Learning One Representation to Optimize All Rewardscited by 115
8 more
  1. 2020C-Learning: Learning to Achieve Goals via Recursive Classificationcited by 103
  2. 2024Foundation Policies with Hilbert Representationscited by 94
  3. 2022Does Zero-Shot Reinforcement Learning Exist?cited by 83
  4. 2024Learning Temporal Distances: Contrastive Successor Features Can Provide a Metric Structure for Decision-Makingcited by 56
  5. 2017Eigendecompositions of Transfer Operators in Reproducing Kernel Hilbert Spacescited by 48
  6. 2025Offline Goal-conditioned Reinforcement Learning with Quasimetric Representationscited by 24
  7. 2024Proto Successor Measure: Representing the Behavior Space of an RL Agentcited by 20
  8. 2023Learning invariant representations of time-homogeneous stochastic dynamical systemscited by 17

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026ChronoSRL: Temporal Geometry for Self-Supervised Reinforcement Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Laplacian Representations for Decision-Time Planning
  2. 2025Structural Information-based Hierarchical Diffusion for Offline Reinforcement Learning
  3. 2025Test-Time Graph Search for Goal-Conditioned Reinforcement Learning
  4. 2025Learning from Reward-Free Offline Data: A Case for Planning with Latent Dynamics Models
  5. 2026Matrix-Space Reinforcement Learning for Reusing Local Transition Geometry
  6. 2023Efficient Planning with Latent Diffusion
  7. 2025Offline Goal-Conditioned Reinforcement Learning with Projective Quasimetric Planning
  8. 2024Temporal Abstraction in Reinforcement Learning with Offline Data