mlsanityData to 14 Oct 2026

ViSA: Visited-State Augmentation for Generalized Goal-Space Contrastive Reinforcement Learning

arXiv 2603.14887 ↗First posted 16 Mar 2026cs.ROcited by 0

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations

Abstract (arXiv)

Goal-Conditioned Reinforcement Learning (GCRL) is a framework for learning a policy that can reach arbitrarily given goals. In particular, Contrastive Reinforcement Learning (CRL) provides a framework for policy updates using an approximation of the value function estimated via contrastive learning, achieving higher sample efficiency compared to conventional methods. However, since CRL treats the visited state as a pseudo-goal during learning, it can accurately estimate the value function only for limited goals. To address this issue, we propose a novel data augmentation approach for CRL called ViSA (Visited-State Augmentation). ViSA consists of two components: 1) generating augmented state samples, with the aim of augmenting hard-to-visit state samples during on-policy exploration, and 2) learning consistent embedding space, which uses an augmented state as auxiliary information to regularize the embedding space by reformulating the objective function of the embedding space based on mutual information. We evaluate ViSA in simulation and real-world robotic tasks and show improved goal-space generalization, which permits accurate value estimation for hard-to-visit goals. Further details can be found on the project page: https://issa-n.github.io/projectPage_ViSA/

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 28 of its 30 references that are in the map. First posted on arXiv 16 Mar 2026.

Builds on

28 of 30 references.

References · 28

  1. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  2. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  3. 2020Supervised Contrastive Learningcited by 2,829
  4. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,546
  5. 2017Hindsight Experience Replaycited by 1,136
  6. 2017Sim-to-Real Transfer of Robotic Control with Dynamics Randomizationcited by 672
  7. 2019On Variational Bounds of Mutual Informationcited by 559
  8. 2020Debiased Contrastive Learningcited by 356
  9. 2019Unsupervised Scalable Representation Learning for Multivariate Time Seriescited by 269
  10. 2022Contrastive Learning as Goal-Conditioned Reinforcement Learningcited by 253
  11. 2019Planning with Goal-Conditioned Policiescited by 149
  12. 2021Goal-Conditioned Reinforcement Learning with Imagined Subgoalscited by 134
16 more
  1. 2021Multimodal Contrastive Training for Visual Representation Learningcited by 124
  2. 2020C-Learning: Learning to Achieve Goals via Recursive Classificationcited by 103
  3. 2023Factorized Contrastive Learning: Going Beyond Multi-view Redundancycited by 75
  4. 2019ROBEL: Robotics Benchmarks for Learning with Low-Cost Robotscited by 72
  5. 2020A Contrastive Learning Approach for Training Variational Autoencoder Priorscited by 66
  6. 2023Soft Contrastive Learning for Time Seriescited by 51
  7. 2024What to align in multimodal contrastive learning?cited by 47
  8. 2023Contrastive Difference Predictive Codingcited by 36
  9. 2022Generalizing Goal-Conditioned Reinforcement Learning with Variational Causal Reasoningcited by 34
  10. 2021Decomposed Mutual Information Estimation for Contrastive Representation Learningcited by 25
  11. 2024A Single Goal is All You Need: Skills and Exploration Emerge from Contrastive RL without Rewards, Demonstrations, or Subgoalscited by 23
  12. 2021MHER: Model-based Hindsight Experience Replaycited by 15
  13. 2023Nonprehensile Planar Manipulation through Reinforcement Learning with Multimodal Categorical Explorationcited by 9
  14. 2023Goal-Conditioned Reinforcement Learning with Disentanglement-based Reachability Planningcited by 5
  15. 2023Unsupervised domain adaptation by learning using privileged informationcited by 4
  16. 2022Goal Exploration Augmentation via Pre-trained Skills for Sparse-Reward Long-Horizon Goal-Conditioned Reinforcement Learningcited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Self-Supervised On-Policy Reinforcement Learning via Contrastive Proximal Policy Optimisation
  2. 2024On-Robot Reinforcement Learning with Goal-Contrastive Rewards
  3. 2025Offline Goal-conditioned Reinforcement Learning with Quasimetric Representations
  4. 2026Do Better Goal Representations Improve Goal-Conditioned Reinforcement Learning?
  5. 2025Equivariant Goal Conditioned Contrastive Reinforcement Learning
  6. 2025Demystifying the Mechanisms Behind Emergent Exploration in Goal-conditioned RL
  7. 2024TLDR: Unsupervised Goal-Conditioned RL via Temporal Distance-Aware Representations
  8. 2023A State Augmentation based approach to Reinforcement Learning from Human Preferences