mlsanityData to 14 Oct 2026

Skill Learning via Policy Diversity Yields Identifiable Representations for Reinforcement Learning

arXiv 2507.14748 ↗First posted 19 Jul 2025cs.LG cs.AI stat.MLcited by 4

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations

Abstract (arXiv)

Self-supervised feature learning and pretraining methods in reinforcement learning (RL) often rely on information-theoretic principles, termed mutual information skill learning (MISL). These methods aim to learn a representation of the environment while also incentivizing exploration thereof. However, the role of the representation and mutual information parametrization in MISL is not yet well understood theoretically. Our work investigates MISL through the lens of identifiable representation learning by focusing on the Contrastive Successor Features (CSF) method. We prove that CSF can provably recover the environment's ground-truth features up to a linear transformation due to the inner product parametrization of the features and skill diversity in a discriminative sense. This first identifiability guarantee for representation learning in RL also helps explain the implications of different mutual information objectives and the downsides of entropy regularizers. We empirically validate our claims in MuJoCo and DeepMind Control and show how CSF provably recovers the ground-truth features both from states and pixels.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 52 of its 73 references that are in the map. First posted on arXiv 19 Jul 2025.

Builds on

52 of 73 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025A Survey of State Representation Learning for Deep Reinforcement Learning20 Jun
  2. 2025Task Adaptation from Skills: Information Geometry, Disentanglement, and New Objectives for Unsupervised Reinforcement Learning12 Jun
  3. 2024Can a MISL Fly? Analysis and Ingredients for Mutual Information Skill Learning11 Dec

Other references · 49

  1. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  2. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  3. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  4. 2020Shortcut Learning in Deep Neural Networkscited by 1,969
  5. 2018Unsupervised Feature Learning via Non-Parametric Instance-level Discriminationcited by 1,845
  6. 2020Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hyperspherecited by 1,405
  7. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,194
  8. 2018Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representationscited by 941
  9. 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 752
  10. 2023The Linear Representation Hypothesis and the Geometry of Large Language Modelscited by 716
  11. 2019Variational Autoencoders and Nonlinear ICA: A Unifying Frameworkcited by 503
  12. 2020Planning to Explore via Self-Supervised World Modelscited by 368
37 more
  1. 2019On Mutual Information Maximization for Representation Learningcited by 316
  2. 2019Dynamics-Aware Unsupervised Discovery of Skillscited by 313
  3. 2018Nonlinear ICA Using Auxiliary Variables and Generalized Contrastive Learningcited by 282
  4. 2018Large-Scale Study of Curiosity-Driven Learningcited by 280
  5. 2016Unsupervised Feature Extraction by Time-Contrastive Learning and Nonlinear ICAcited by 273
  6. 2016Variational Intrinsic Controlcited by 267
  7. 2019Self-Supervised Exploration via Disagreementcited by 265
  8. 2022Contrastive Learning as Goal-Conditioned Reinforcement Learningcited by 253
  9. 2020Weakly-Supervised Disentanglement Without Compromisescited by 239
  10. 2021Contrastive Learning Inverts the Data Generating Processcited by 221
  11. 2021Towards Causal Representation Learningcited by 209
  12. 2024The Geometry of Categorical and Hierarchical Concepts in Large Language Modelscited by 159
  13. 2015Variational Information Maximisation for Intrinsically Motivated Reinforcement Learningcited by 144
  14. 2019Fast Task Inference with Variational Intrinsic Successor Featurescited by 133
  15. 2022CITRIS: Causal Identifiability from Temporal Intervened Sequencescited by 133
  16. 2018Variational Option Discovery Algorithmscited by 122
  17. 2020On Linear Identifiability of Learned Representationscited by 105
  18. 2022Chaos is a Ladder: A New Theoretical Understanding of Contrastive Learning via Augmentation Overlapcited by 105
  19. 2018Unsupervised Control Through Non-Parametric Discriminative Rewardscited by 102
  20. 2020ICE-BeeM: Identifiable Conditional Energy-Based Deep Models Based on Nonlinear ICAcited by 102
  21. 2019The Incomplete Rosetta Stone Problem: Identifiability Results for Multi-View Nonlinear ICAcited by 86
  22. 2023METRA: Scalable Unsupervised RL with Metric-Aware Abstractioncited by 86
  23. 2021Disentangling Identifiable Features from Noisy Data with Structured Nonlinear ICAcited by 64
  24. 2022Lipschitz-constrained Unsupervised Skill Discoverycited by 62
  25. 2020CausalVAE: Structured Causal Disentanglement in Variational Autoencodercited by 48
  26. 2024InfoNCE: Identifying the Gap Between Theory and Practicecited by 40
  27. 2022Self-Supervised Learning via Maximum Entropy Codingcited by 33
  28. 2023Optimistic Active Exploration of Dynamical Systemscited by 32
  29. 2024Cross-Entropy Is All You Need To Invert the Data Generating Processcited by 31
  30. 2023ALAN: Autonomously Exploring Robotic Agents in the Real Worldcited by 24
  31. 2024A Single Goal is All You Need: Skills and Exploration Emerge from Contrastive RL without Rewards, Demonstrations, or Subgoalscited by 23
  32. 2023Causal Triplet: An Open Challenge for Intervention-centric Causal Representation Learningcited by 22
  33. 2020Independent Innovation Analysis for Nonlinear Vector Autoregressive Processcited by 19
  34. 2024Identifiable Exchangeable Mechanisms for Causal Structure and Representation Learningcited by 16
  35. 2022What Do We Maximize in Self-Supervised Learning?cited by 15
  36. 2021Deep Data Density Estimation through Donsker-Varadhan Representationcited by 6
  37. 2023Regularity as Intrinsic Reward for Free Playcited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 3.

4 papers.

  1. 2025VendiRL: A Framework for Self-Supervised Reinforcement Learning of Diversely Diverse Skillscited by 2
  2. 2026Unifying Goal-Conditioned RL and Unsupervised Skill Learning via Control-Maximizationcited by 2
  3. 2026On the Emergence and Test-Time Use of Structural Information in Large Language Modelscited by 1
  4. 2026Learning to Perceive the World Through Control: Empowerment-Based Representation Learningcited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2024Skill-aware Mutual Information Optimisation for Generalisation in Reinforcement Learning
  2. 2025Unsupervised Skill Discovery through Skill Regions Differentiation
  3. 2025Representation Convergence: Mutual Distillation is Secretly a Form of Regularization
  4. 2023Offline Diversity Maximization Under Imitation Constraints
  5. 2022Robust Task Representations for Offline Meta-Reinforcement Learning via Contrastive Learning
  6. 2025Dynamic Contrastive Skill Learning with State-Transition Based Skill Clustering and Dynamic Length Adjustment
  7. 2024Learning Successor Features the Simple Way
  8. 2024Robust Policy Learning via Offline Skill Diffusion