mlsanityData to 14 Oct 2026

Learning Generalizable Skill Policy with Data-Efficient Unsupervised RL

arXiv 2607.00392 ↗First posted 1 Jul 2026cs.LG cs.AIcited by 1

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations

Abstract (arXiv)

Unsupervised Reinforcement Learning (URL) aims to pre-train scalable, skill-conditioned policies without extrinsic rewards, serving as a foundation for downstream control tasks. Despite recent progress, we argue that current off-policy URL methods are limited by two critical, overlooked bottlenecks: (1) non-stationary skill semantics and (2) brittle generalization. To address these challenges, we propose GenDa (Generalizable Data-efficient Agent), a unified framework for robust unsupervised reinforcement learning. First, we introduce a skill relabeling mechanism to mitigate non-stationarity and significantly improve data efficiency for pre-training. Second, we propose a Complementary Information Bottleneck (CIB), encouraging the learned skill policy to focus on ego-centric features and become robust to distribution shifts for downstream tasks. Through various experiments, we demonstrate that GenDa significantly enhances the scalability of URL with superior generalizability and data efficiency. Our code and videos are available at https://ihatebroccoli.github.io/official-GenDa.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 35 of its 39 references that are in the map. First posted on arXiv 1 Jul 2026.

Builds on

35 of 39 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025Periodic Skill Discovery5 Nov
  2. 2025Task Adaptation from Skills: Information Geometry, Disentanglement, and New Objectives for Unsupervised Reinforcement Learning12 Jun
  3. 2024Can a MISL Fly? Analysis and Ingredients for Mutual Information Skill Learning11 Dec
  4. 2024Learning Temporal Distances: Contrastive Successor Features Can Provide a Metric Structure for Decision-Making24 Jun

Other references · 31

  1. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  3. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 2,026
  4. 2017Hindsight Experience Replaycited by 1,136
  5. 2018DeepMind Control Suitecited by 956
  6. 2016Deep Variational Information Bottleneckcited by 930
  7. 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 752
  8. 2019Data-Efficient Image Recognition with Contrastive Predictive Codingcited by 673
  9. 2019Relay Policy Learning: Solving Long-Horizon Tasks via Imitation and Reinforcement Learningcited by 384
  10. 2021Mastering Visual Continuous Control: Improved Data-Augmented Reinforcement Learningcited by 372
  11. 2020Data-Efficient Reinforcement Learning with Self-Predictive Representationscited by 318
  12. 2019Dynamics-Aware Unsupervised Discovery of Skillscited by 313
19 more
  1. 2016Variational Intrinsic Controlcited by 267
  2. 2021Behavior From the Void: Unsupervised Active Pre-Trainingcited by 174
  3. 2019Fast Task Inference with Variational Intrinsic Successor Featurescited by 133
  4. 2021Discovering and Achieving Goals via World Modelscited by 131
  5. 2020Explore, Discover and Learn: Unsupervised Discovery of State-Covering Skillscited by 122
  6. 2018Noise Contrastive Estimation and Negative Sampling for Conditional Models: Consistency and Statistical Efficiencycited by 105
  7. 2018Unsupervised Control Through Non-Parametric Discriminative Rewardscited by 102
  8. 2023METRA: Scalable Unsupervised RL with Metric-Aware Abstractioncited by 86
  9. 2019Wasserstein Dependency Measure for Representation Learningcited by 69
  10. 2019Dynamical Distance Learning for Semi-Supervised and Unsupervised Skill Discoverycited by 68
  11. 2022Lipschitz-constrained Unsupervised Skill Discoverycited by 62
  12. 2023Controllability-Aware Unsupervised Skill Discoverycited by 49
  13. 2021Learning more skills through optimistic explorationcited by 38
  14. 2021Unsupervised Skill Discovery with Bottleneck Option Learningcited by 28
  15. 2023Behavior Contrastive Learning for Unsupervised Skill Discoverycited by 26
  16. 2021Wasserstein Unsupervised Reinforcement Learningcited by 22
  17. 2022Mastering the Unsupervised Reinforcement Learning Benchmark from Pixelscited by 21
  18. 2021Direct then Diffuse: Incremental Unsupervised Skill Discovery for State Covering and Goal Reachingcited by 19
  19. 2022A Mixture of Surprises for Unsupervised Reinforcement Learningcited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Scaling Empowered Agentscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Foundation Policies with Hilbert Representations
  2. 2024Unsupervised-to-Online Reinforcement Learning
  3. 2024Robust Policy Learning via Offline Skill Diffusion
  4. 2025Unsupervised Skill Discovery through Skill Regions Differentiation
  5. 2024Constrained Ensemble Exploration for Unsupervised Skill Discovery
  6. 2026Paying Less Generalization Tax: A Cross-Domain Generalization Study of RL Training for LLM Agents
  7. 2022Domain Generalization for Robust Model-Based Offline Reinforcement Learning
  8. 2026Unsupervised Learning of Efficient Exploration: Pre-training Adaptive Policies via Self-Imposed Goals