mlsanityData to 14 Oct 2026

ChronoSRL: Temporal Geometry for Self-Supervised Reinforcement Learning

arXiv 2609.36238 ↗First posted 28 Sep 2026cs.AI cs.ROcited by 0

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations

Abstract (arXiv)

A goal that is close in space can be far away in time. Obstacles, terrain, and the agent's own capabilities determine how long it takes to get there. Yet, critics in contrastive and survival reinforcement learning do not measure the distances in their representation space in units of time. We therefore introduce ChronoSRL, which gives the critic's embeddings an explicit temporal geometry. The distance between state-action and goal embeddings is trained to match the time that the agent takes to reach the goal (goal-reaching time), while goals that were not reached, and goals from other trajectories, are pushed at least one discount horizon away. Furthermore, reaching a goal quickly once does not mean that reaching it is reliable in general, so the policy should not follow the temporal distance directly. Instead, we build on survival reinforcement learning and predict from our temporal embeddings not only the full distribution of goal-reaching times but also the time spent near the goal. Thereby, the policy is trained to favor actions that reach the goal sooner and more reliably and that keep the agent near it. ChronoSRL learns faster and reaches higher performance than contrastive, action-chunked contrastive, and survival reinforcement learning baselines on seven standard locomotion and navigation benchmarks, even with much smaller networks. To test the limits of self-supervised reinforcement learning, we introduce velocity tracking, goal-position reaching, and box climbing tasks with a quadruped robot in a realistic sim-to-real locomotion setup, and show how the shaping terms that are typical for robotics can be naturally incorporated into our framework. ChronoSRL is the only one of the tested self-supervised reinforcement learning methods that learns to stay at the commanded velocities and goal positions, and climbs the highest boxes.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 39 of its 53 references that are in the map. First posted on arXiv 28 Sep 2026.

Builds on

39 of 53 references.

Closest ancestors · 13

Same direction, newest first

  1. 2026Three Steps at a Time: Learning Representations from Action Sequences in Contrastive RL31 Aug
  2. 2026Survival Reinforcement Learning: Toward Scalable Self-Supervised RL29 May
  3. 2026Hitting Time Isomorphism for Multi-Stage Planning with Foundation Policies7 May
  4. 2026SVL: Goal-Conditioned Reinforcement Learning as Survival Learning19 Apr
  5. 2025Multistep Quasimetric Learning for Scalable Goal-conditioned Reinforcement Learning11 Nov
  6. 2025XQC: Well-conditioned Optimization Accelerates Deep Reinforcement Learning29 Sep
  7. 2025Reinforcement Learning with Action Chunking10 Jul
  8. 20251000 Layer Networks for Self-Supervised RL: Scaling Depth Can Enable New Goal-Reaching Capabilities19 Mar
  9. 2024SimBa: Simplicity Bias for Scaling Up Parameters in Deep Reinforcement Learning13 Oct
  10. 2024TLDR: Unsupervised Goal-Conditioned RL via Temporal Distance-Aware Representations11 Jul
  11. 2024Learning Temporal Distances: Contrastive Successor Features Can Provide a Metric Structure for Decision-Making24 Jun
  12. 2024Bigger, Regularized, Optimistic: scaling for compute and sample-efficient continuous control25 May
1 more
  1. 2024Foundation Policies with Hilbert Representations23 Feb

Other references · 26

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  3. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  4. 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,568
  5. 2021Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learningcited by 512
  6. 2016DeepSurv: Personalized Treatment Recommender System Using A Cox Proportional Hazards Deep Neural Networkcited by 358
  7. 2016Successor Features for Transfer in Reinforcement Learningcited by 340
  8. 2022Contrastive Learning as Goal-Conditioned Reinforcement Learningcited by 253
  9. 2019Time-to-Event Prediction with Neural Networks and Cox Regressioncited by 175
  10. 2022Goal-Conditioned Reinforcement Learning: Problems and Solutionscited by 158
  11. 2023HIQL: Offline Goal-Conditioned RL with Latent States as Actionscited by 141
  12. 2021Discovering and Achieving Goals via World Modelscited by 131
14 more
  1. 2023Optimal Goal-Reaching Reinforcement Learning via Quasimetric Learningcited by 121
  2. 2020C-Learning: Learning to Achieve Goals via Recursive Classificationcited by 103
  3. 2020Maximum Entropy Gain Exploration for Long Horizon Multi-goal Reinforcement Learningcited by 102
  4. 2023METRA: Scalable Unsupervised RL with Metric-Aware Abstractioncited by 86
  5. 2018A Scalable Discrete-Time Survival Model for Neural Networkscited by 72
  6. 2019Dynamical Distance Learning for Semi-Supervised and Unsupervised Skill Discoverycited by 68
  7. 2024One Policy to Run Them All: an End-to-end Learning Approach to Multi-Embodiment Locomotioncited by 43
  8. 2023Contrastive Difference Predictive Codingcited by 36
  9. 2025Offline Goal-conditioned Reinforcement Learning with Quasimetric Representationscited by 24
  10. 2023Stabilizing Contrastive RL: Techniques for Robotic Goal Reaching from Offline Datacited by 23
  11. 2019Self-supervised Learning of Distance Functions for Goal-Conditioned Reinforcement Learningcited by 18
  12. 2019Self-supervised Learning of Image Embedding for Continuous Controlcited by 16
  13. 2022On the Learning and Learnability of Quasimetricscited by 15
  14. 2023RL-X: A Deep Reinforcement Learning Library (not only) for RoboCupcited by 8

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Learning When to Act: Interval-Aware Reinforcement Learning with Predictive Temporal Structure
  2. 2026How Long, Not How Close: A Learned Temporal Metric for Planning in Latent World Models
  3. 2026Temporal Representations for Exploration: Learning Complex Exploratory Behavior without Extrinsic Rewards
  4. 2024State Chrono Representation for Enhancing Generalization in Reinforcement Learning
  5. 2026Learning Multiple Timescales for Goal-Conditioned Reinforcement Learning
  6. 2026Chronos Policy: Learning Robot Policies on Physical Time
  7. 2026Do Better Goal Representations Improve Goal-Conditioned Reinforcement Learning?
  8. 2025Test-Time Graph Search for Goal-Conditioned Reinforcement Learning