Saliency-Guided Representation with Consistency Policy Learning for Visual Unsupervised Reinforcement Learning
Not on the map. Why?
Abstract (arXiv)
Zero-shot unsupervised reinforcement learning (URL) offers a promising direction for building generalist agents capable of generalizing to unseen tasks without additional supervision. Among existing approaches, successor representations (SR) have emerged as a prominent paradigm due to their effectiveness in structured, low-dimensional settings. However, SR methods struggle to scale to high-dimensional visual environments. Through empirical analysis, we identify two key limitations of SR in visual URL: (1) SR objectives often lead to suboptimal representations that attend to dynamics-irrelevant regions, resulting in inaccurate successor measures and degraded task generalization; and (2) these flawed representations hinder SR policies from modeling multi-modal skill-conditioned action distributions and ensuring skill controllability. To address these limitations, we propose Saliency-Guided Representation with Consistency Policy Learning (SRCP), a novel framework that improves zero-shot generalization of SR methods in visual URL. SRCP decouples representation learning from successor training by introducing a saliency-guided dynamics task to capture dynamics-relevant representations, thereby improving successor measure and task generalization. Moreover, it integrates a fast-sampling consistency policy with URL-specific classifier-free guidance and tailored training objectives to improve skill-conditioned policy modeling and controllability. Extensive experiments on 16 tasks across 4 datasets from the ExORL benchmark demonstrate that SRCP achieves state-of-the-art zero-shot generalization in visual URL and is compatible with various SR methods.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 40 of its 50 references that are in the map. First posted on arXiv 7 Apr 2026.
Builds on
40 of 50 references.
References · 40
- 2020Denoising Diffusion Probabilistic Modelscited by 20,851
- 2023Consistency Modelscited by 1,778
- 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,653
- 2018Exploration by Random Network Distillationcited by 966
- 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 734
- 2020dm_control: Software and Tasks for Continuous Controlcited by 368
- 2019Dynamics-Aware Unsupervised Discovery of Skillscited by 308
- 2019Self-Supervised Exploration via Disagreementcited by 263
- 2023Imitating Human Behaviour with Diffusion Modelscited by 260
- 2019Efficient Exploration via State Marginal Matchingcited by 186
- 2021Reinforcement Learning with Prototypical Representationscited by 179
- 2022Offline Reinforcement Learning via High-Fidelity Generative Behavior Modelingcited by 177
28 more
- 2021Behavior From the Void: Unsupervised Active Pre-Trainingcited by 173
- 2023Efficient Diffusion Policies for Offline Reinforcement Learningcited by 151
- 2022Contrastive and Non-Contrastive Self-Supervised Learning Recover Global and Local Spectral Embedding Methodscited by 146
- 2021URLB: Unsupervised Reinforcement Learning Benchmarkcited by 138
- 2021Learning One Representation to Optimize All Rewardscited by 110
- 2021APS: Active Pretraining with Successor Featurescited by 109
- 2022Don't Change the Algorithm, Change the Data: Exploratory Data for Offline Reinforcement Learningcited by 93
- 2023Consistency Models as a Rich and Efficient Policy Class for Reinforcement Learningcited by 93
- 2024Foundation Policies with Hilbert Representationscited by 89
- 2018The Laplacian in RL: Learning Representations with Efficient Approximationscited by 83
- 2023METRA: Scalable Unsupervised RL with Metric-Aware Abstractioncited by 80
- 2022Does Zero-Shot Reinforcement Learning Exist?cited by 79
- 2022Lipschitz-constrained Unsupervised Skill Discoverycited by 60
- 2023TACO: Temporal Latent Action-Driven Contrastive Loss for Visual Reinforcement Learningcited by 56
- 2020Variational Autoencoders for Learning Nonlinear Dynamics of Physical Systemscited by 50
- 2021The Information Geometry of Unsupervised Reinforcement Learningcited by 36
- 2023Boosting Continuous Control with Consistency Policycited by 36
- 2022Spectral Decomposition Representation for Reinforcement Learningcited by 35
- 2022TrajGen: Generating Realistic and Diverse Trajectories with Reactive and Feasible Agent Behaviors for Autonomous Drivingcited by 23
- 2025ReasonPlan: Unified Scene Prediction and Decision Reasoning for Closed-loop Autonomous Drivingcited by 22
- 2023Zero-Shot Reinforcement Learning from Low Quality Datacited by 21
- 2024Generalizing Consistency Policy to Visual RL with Prioritized Proximal Experience Regularizationcited by 11
- 2024Premier-TACO is a Few-Shot Policy Learner: Pretraining Multitask Representation via Temporal Action-Driven Contrastive Losscited by 10
- 2025Perception-Consistency Multimodal Large Language Models Reasoning via Caption-Regularized Policy Optimizationcited by 9
- 2024Constrained Ensemble Exploration for Unsupervised Skill Discoverycited by 7
- 2025Task Adaptation from Skills: Information Geometry, Disentanglement, and New Objectives for Unsupervised Reinforcement Learningcited by 5
- 2025Videos are Sample-Efficient Supervisions: Behavior Cloning from Videos via Latent Representationscited by 4
- 2025Equilibrium Policy Generalization: A Reinforcement Learning Framework for Cross-Graph Zero-Shot Generalization in Pursuit-Evasion Gamescited by 3
Built on by
Citing papers per year: 2026: 1.
1 paper.