mlsanityData to 14 Oct 2026

Salience-Invariant Consistent Policy Learning for Generalization in Visual Reinforcement Learning

arXiv 2502.08336 ↗First posted 12 Feb 2025cs.AIcited by 1

On the map: Deep reinforcement learning methods → RL for robot locomotion and manipulation

Abstract (arXiv)

Generalizing policies to unseen scenarios remains a critical challenge in visual reinforcement learning, where agents often overfit to the specific visual observations of the training environment. In unseen environments, distracting pixels may lead agents to extract representations containing task-irrelevant information. As a result, agents may deviate from the optimal behaviors learned during training, thereby hindering visual generalization.To address this issue, we propose the Salience-Invariant Consistent Policy Learning (SCPL) algorithm, an efficient framework for zero-shot generalization. Our approach introduces a novel value consistency module alongside a dynamics module to effectively capture task-relevant representations. The value consistency module, guided by saliency, ensures the agent focuses on task-relevant pixels in both original and perturbed observations, while the dynamics module uses augmented data to help the encoder capture dynamic- and reward-relevant representations. Additionally, our theoretical analysis highlights the importance of policy consistency for generalization. To strengthen this, we introduce a policy consistency module with a KL divergence constraint to maintain consistent policies across original and perturbed observations.Extensive experiments on the DMC-GB, Robotic Manipulation, and CARLA benchmarks demonstrate that SCPL significantly outperforms state-of-the-art methods in terms of generalization. Notably, SCPL achieves average performance improvements of 14%, 39%, and 69% in the challenging DMC video hard setting, the Robotic hard setting, and the CARLA benchmark, respectively.Project Page: https://sites.google.com/view/scpl-rl.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods. Builds on 26 of its 42 references that are in the map. First posted on arXiv 12 Feb 2025.

Builds on

26 of 42 references.

References · 26

  1. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  2. 2015Trust Region Policy Optimizationcited by 2,580
  3. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,371
  4. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,353
  5. 2018DeepMind Control Suitecited by 956
  6. 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 585
  7. 2020Reinforcement Learning with Augmented Datacited by 468
  8. 2020Learning Invariant Representations for Reinforcement Learning without Reconstructioncited by 436
  9. 2019Improving Sample Efficiency in Model-Free Reinforcement Learning from Imagescited by 274
  10. 2020Generalization in Reinforcement Learning by Soft Data Augmentationcited by 137
  11. 2020Self-Supervised Policy Adaptation during Deploymentcited by 123
  12. 2020Phasic Policy Gradientcited by 123
14 more
  1. 2021Stabilizing Deep Q-Learning with ConvNets and Vision Transformers under Data Augmentationcited by 105
  2. 2019Network Randomization: A Simple Technique for Generalization in Deep Reinforcement Learningcited by 104
  3. 2021Decoupling Value and Policy for Generalization in Reinforcement Learningcited by 82
  4. 2022Pre-Trained Image Encoder for Generalizable Visual Reinforcement Learningcited by 60
  5. 2022Iso-Dream: Isolating and Leveraging Noncontrollable Visual Dynamics in World Modelscited by 53
  6. 2022Look Closer: Bridging Egocentric and Third-Person Views with Transformers for Robotic Manipulationcited by 49
  7. 2021SECANT: Self-Expert Cloning for Zero-Shot Generalization of Visual Policiescited by 43
  8. 2020Deep Reinforcement Learning based Automatic Exploration for Navigation in Unknown Environmentcited by 34
  9. 2022Look where you look! Saliency-guided Q-networks for generalization in visual Reinforcement Learningcited by 34
  10. 2022Does Self-supervised Learning Really Improve Reinforcement Learning from Pixels?cited by 24
  11. 2022Don't Touch What Matters: Task-Aware Lipschitz Data Augmentation for Visual Reinforcement Learningcited by 21
  12. 2022Temporal Disentanglement of Representations for Improved Generalisation in Reinforcement Learningcited by 19
  13. 2023Improving Generalization in Visual Reinforcement Learning via Conflict-aware Gradient Agreement Augmentationcited by 14
  14. 2022Rethinking Value Function Learning for Generalization in Reinforcement Learningcited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026What to Ignore, What to React: Visually Robust RL Fine-Tuning of VLA Modelscited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2026Saliency-Guided Representation with Consistency Policy Learning for Visual Unsupervised Reinforcement Learning
  2. 2026VIGOR: Zero-Shot Visual Generalization via Latent-Space Consistency in Model-Based Reinforcement Learning
  3. 2024Focus On What Matters: Separated Models For Visual-Based RL Generalization
  4. 2025Pre-trained Visual Representations Generalize Where it Matters in Model-Based Reinforcement Learning
  5. 2024Generalizing Consistency Policy to Visual RL with Prioritized Proximal Experience Regularization
  6. 2025Representation Convergence: Mutual Distillation is Secretly a Form of Regularization
  7. 2024ViSaRL: Visual Reinforcement Learning Guided by Human Saliency
  8. 2024Inferring Behavior-Specific Context Improves Zero-Shot Generalization in Reinforcement Learning