mlsanityData to 14 Oct 2026

MSPR: Multi-scale Predictive Representations for Goal-conditioned Reinforcement Learning

arXiv 2605.09364 ↗First posted 10 May 2026cs.LGcited by 1

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations

Abstract (arXiv)

This paper investigates robust representation learning in offline goal-conditioned reinforcement learning (GCRL). Particularly in sparse reward scenarios, learning representations that align state and goal latents is a challenge, as the encoder can learn goal-agnostic features that destabilize policy learning. We address this issue by learning the encoder's representation with alignment objectives that capture the environment across multiple scales, from local physical dynamics to long-horizon goal-directed structure. Concretely, we propose MSPR, a framework that leverages multi-scale predictive supervision to enforce goal-directed alignment within the latent space. We demonstrate that MSPR leads to strong performance on both vision and state-based tasks. Furthermore, we show that our approach is resilient under realistic, challenging data regimes, maintaining state-of-the-art performance across a wide variety of tasks.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 38 of its 55 references that are in the map. First posted on arXiv 10 May 2026.

Builds on

38 of 55 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025Dual Goal Representations8 Oct
  2. 2025A Survey of State Representation Learning for Deep Reinforcement Learning20 Jun
  3. 2025Towards General-Purpose Model-Free Reinforcement Learning27 Jan
  4. 2024Q-WSL: Optimizing Goal-Conditioned RL with Weighted Supervised Learning via Dynamic Programming9 Oct

Other references · 34

  1. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  2. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,809
  3. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,653
  4. 2018World Modelscited by 1,513
  5. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,353
  6. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,293
  7. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,264
  8. 2023Mastering Diverse Domains through World Modelscited by 1,253
  9. 2017Hindsight Experience Replaycited by 1,136
  10. 2021A Minimalist Approach to Offline Reinforcement Learningcited by 899
  11. 2018IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architecturescited by 883
  12. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 760
22 more
  1. 2024Revisiting Feature Prediction for Learning Visual Representations from Videocited by 531
  2. 2023TD-MPC2: Scalable, Robust World Models for Continuous Controlcited by 512
  3. 2022VIP: Towards Universal Visual Reward and Representation via Value-Implicit Pre-Trainingcited by 395
  4. 2020Data-Efficient Reinforcement Learning with Self-Predictive Representationscited by 318
  5. 2019Learning Latent Plans from Playcited by 316
  6. 2022Contrastive Learning as Goal-Conditioned Reinforcement Learningcited by 253
  7. 2024OGBench: Benchmarking Offline Goal-Conditioned RLcited by 238
  8. 2019DeepMDP: Learning Continuous Latent Space Models for Representation Learningcited by 207
  9. 2023HIQL: Offline Goal-Conditioned RL with Latent States as Actionscited by 141
  10. 2019Scalable methods for computing state similarity in deterministic Markov Decision Processescited by 133
  11. 2023Optimal Goal-Reaching Reinforcement Learning via Quasimetric Learningcited by 121
  12. 2020C-Learning: Learning to Achieve Goals via Recursive Classificationcited by 103
  13. 2022Deep Hierarchical Planning from Pixelscited by 101
  14. 2022Rethinking Goal-conditioned Supervised Learning and Its Connection to Offline RLcited by 89
  15. 2024Is Value Learning Really the Main Bottleneck in Offline RL?cited by 89
  16. 2023For SALE: State-Action Representation Learning for Deep Reinforcement Learningcited by 75
  17. 2022How Far I'll Go: Offline Goal-Conditioned Reinforcement Learning via f-Advantage Regressioncited by 66
  18. 2021MICo: Improved representations via sampling-based state similarity for Markov decision processescited by 43
  19. 2018Deep Reinforcement Learning for Vision-Based Robotic Grasping: A Simulated Comparative Evaluation of Off-Policy Methodscited by 33
  20. 2025Temporal Representation Alignment: Successor Features Enable Emergent Compositionality in Robot Instruction Followingcited by 14
  21. 2025Self-Predictive Representations for Combinatorial Generalization in Behavioral Cloningcited by 12
  22. 2023ReRoGCRL: Representation-based Robustness in Goal-Conditioned Reinforcement Learningcited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Learning Multiple Timescales for Goal-Conditioned Reinforcement Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Latent Representation Alignment for Offline Goal-Conditioned Reinforcement Learning
  2. 2026Do Better Goal Representations Improve Goal-Conditioned Reinforcement Learning?
  3. 2026Action-Sufficient Goal Representations
  4. 2026Provable Multi-Task Reinforcement Learning: A Representation Learning Framework with Low Rank Rewards
  5. 2026Dual Advantage Fields
  6. 2026DAGR: State-Conditioned Goal Representations via Difference-Aware Goal Cross-Attention
  7. 2024TLDR: Unsupervised Goal-Conditioned RL via Temporal Distance-Aware Representations
  8. 2026Sparse-TD-JEPA: Improving Zero-Shot Reinforcement Learning With Sparse Latent Task Representations