mlsanityData to 7 Oct 2026

CIC: Contrastive Intrinsic Control for Unsupervised Skill Discovery

arXiv 2202.00161 ↗First posted 1 Feb 2022cs.LG cs.AIcited by 66

Not on the map. Why?

Abstract (arXiv)

We introduce Contrastive Intrinsic Control (CIC), an algorithm for unsupervised skill discovery that maximizes the mutual information between state-transitions and latent skill vectors. CIC utilizes contrastive learning between state-transitions and skills to learn behavior embeddings and maximizes the entropy of these embeddings as an intrinsic reward to encourage behavioral diversity. We evaluate our algorithm on the Unsupervised Reinforcement Learning Benchmark, which consists of a long reward-free pre-training phase followed by a short adaptation phase to downstream tasks with extrinsic rewards. CIC substantially improves over prior methods in terms of adaptation efficiency, outperforming prior unsupervised skill discovery methods by 1.79x and the next leading overall exploration algorithm by 1.18x.

Key facts. Cited by 66 papers in the map (data through 7 Oct 2026). Builds on 55 of its 74 references that are in the map. First posted on arXiv 1 Feb 2022.

Builds on

55 of 74 references.

References · 55

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2017Proximal Policy Optimization Algorithmscited by 14,441
  4. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  5. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  6. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  7. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  8. 2015Continuous control with deep reinforcement learningcited by 3,279
  9. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,845
  10. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 1,916
  11. 2016OpenAI Gymcited by 1,684
  12. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
38 more
  1. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,221
  2. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,173
  3. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,084
  4. 2018Exploration by Random Network Distillationcited by 966
  5. 2018DeepMind Control Suitecited by 935
  6. 2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 781
  7. 2017Rainbow: Combining Improvements in Deep Reinforcement Learningcited by 774
  8. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 745
  9. 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 734
  10. 2019Solving Rubik's Cube with a Robot Handcited by 699
  11. 2017Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrationscited by 685
  12. 2019Data-Efficient Image Recognition with Contrastive Predictive Codingcited by 672
  13. 2018Learning Dexterous In-Hand Manipulationcited by 643
  14. 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 573
  15. 2019On Variational Bounds of Mutual Informationcited by 550
  16. 2020Reinforcement Learning with Augmented Datacited by 452
  17. 2016Reinforcement Learning with Unsupervised Auxiliary Taskscited by 400
  18. 2021Mastering Visual Continuous Control: Improved Data-Augmented Reinforcement Learningcited by 359
  19. 2016Successor Features for Transfer in Reinforcement Learningcited by 331
  20. 2019On Mutual Information Maximization for Representation Learningcited by 315
  21. 2020Data-Efficient Reinforcement Learning with Self-Predictive Representationscited by 309
  22. 2019Dynamics-Aware Unsupervised Discovery of Skillscited by 308
  23. 2018Large-Scale Study of Curiosity-Driven Learningcited by 277
  24. 2019Improving Sample Efficiency in Model-Free Reinforcement Learning from Imagescited by 268
  25. 2019Self-Supervised Exploration via Disagreementcited by 263
  26. 2020Decoupling Representation Learning from Reinforcement Learningcited by 263
  27. 2016Variational Intrinsic Controlcited by 262
  28. 2019Efficient Exploration via State Marginal Matchingcited by 186
  29. 2021Reinforcement Learning with Prototypical Representationscited by 179
  30. 2021Behavior From the Void: Unsupervised Active Pre-Trainingcited by 173
  31. 2019Fast Task Inference with Variational Intrinsic Successor Featurescited by 131
  32. 2017Stochastic Neural Networks for Hierarchical Reinforcement Learningcited by 124
  33. 2018Variational Option Discovery Algorithmscited by 121
  34. 2020Explore, Discover and Learn: Unsupervised Discovery of State-Covering Skillscited by 120
  35. 2021APS: Active Pretraining with Successor Featurescited by 109
  36. 2021State Entropy Maximization with Random Encoders for Efficient Explorationcited by 107
  37. 2021Pretraining Representations for Data-Efficient Reinforcement Learningcited by 105
  38. 2018Unsupervised Control Through Non-Parametric Discriminative Rewardscited by 102

50 most cited of 55 All 55 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 21; 2023: 18; 2024: 11; 2025: 9; 2026: 7.

66 papers.

Sort:

By area:

  1. 2022Contrastive Learning as Goal-Conditioned Reinforcement Learningcited by 241
  2. 2022Deep Hierarchical Planning from Pixelscited by 96
  3. 2023Bootstrap Your Own Skills: Learning to Solve New Tasks with Large Language Model Guidancecited by 85
  4. 2022Does Zero-Shot Reinforcement Learning Exist?cited by 79
  5. 2022Reincarnating Reinforcement Learning: Reusing Prior Computation to Accelerate Progresscited by 63
  6. 2022Pre-Trained Image Encoder for Generalizable Visual Reinforcement Learningcited by 58
  7. 2025Zero-Shot Whole-Body Humanoid Control via Behavioral Foundation Modelscited by 51
  8. 2022Masked Autoencoding for Scalable and Generalizable Decision Makingcited by 44
  9. 2023RLCD: Reinforcement Learning from Contrastive Distillation for Language Model Alignmentcited by 42
  10. 2022Choreographer: Learning and Adapting Skills in Imaginationcited by 31
  11. 2024Unsupervised Zero-Shot Reinforcement Learning via Functional Reward Encodingscited by 31
  12. 2023Deep Laplacian-based Options for Temporally-Extended Explorationcited by 28
38 more
  1. 2022Neuroevolution is a Competitive Alternative to Reinforcement Learning for Skill Discoverycited by 23
  2. 2022A Comprehensive Survey of Data Augmentation in Visual Reinforcement Learningcited by 23
  3. 2022Pretraining in Deep Reinforcement Learning: A Surveycited by 18
  4. 2024Can a MISL Fly? Analysis and Ingredients for Mutual Information Skill Learningcited by 18
  5. 2023Outcome-directed Reinforcement Learning by Uncertainty & Temporal Distance-Aware Curriculum Goal Generationcited by 14
  6. 2023LiFT: Unsupervised Reinforcement Learning with Foundation Models as Teacherscited by 14
  7. 2024EXTRACT: Efficient Policy Learning by Extracting Transferable Robot Skills from Offline Datacited by 13
  8. 2024Leveraging Skills from Unlabeled Prior Data for Efficient Online Explorationcited by 13
  9. 2023A Survey of Demonstration Learningcited by 12
  10. 2024Learning to Assist Humans without Inferring Rewardscited by 12
  11. 2022One After Another: Learning Incremental Skills for a Changing Worldcited by 10
  12. 2022A Mixture of Surprises for Unsupervised Reinforcement Learningcited by 9
  13. 2023Cross-domain Random Pre-training with Prototypes for Reinforcement Learningcited by 9
  14. 2023ContraBAR: Contrastive Bayes-Adaptive Deep RLcited by 9
  15. 2024Disentangled Unsupervised Skill Discovery for Efficient Hierarchical Reinforcement Learningcited by 9
  16. 2024SkiLD: Unsupervised Skill Discovery Guided by Factor Interactionscited by 9
  17. 2022First Contact: Unsupervised Human-Machine Co-Adaptation via Mutual Information Maximizationcited by 8
  18. 2022EUCLID: Towards Efficient Unsupervised Reinforcement Learning with Multi-choice Dynamics Modelcited by 8
  19. 2023Learning to Discover Skills through Guidancecited by 8
  20. 2024Do's and Don'ts: Learning Desirable Skills with Instruction Videoscited by 8
  21. 2023Adapt On-the-Go: Behavior Modulation for Single-Life Robot Deploymentcited by 7
  22. 2022Simple Emergent Action Representations from Multi-Task Policy Trainingcited by 6
  23. 2023Leveraging Jumpy Models for Planning and Fast Learning in Robotic Domainscited by 6
  24. 2023Successor-Predecessor Intrinsic Explorationcited by 6
  25. 2025Epistemically-guided forward-backward explorationcited by 6
  26. 2023PASTA: Pretrained Action-State Transformer Agentscited by 5
  27. 2025Task Adaptation from Skills: Information Geometry, Disentanglement, and New Objectives for Unsupervised Reinforcement Learningcited by 5
  28. 2022Sampling Through the Lens of Sequential Decision Makingcited by 4
  29. 2022Skill-Based Reinforcement Learning with Intrinsic Reward Matchingcited by 4
  30. 2022Curiosity in Hindsight: Intrinsic Exploration in Stochastic Environmentscited by 4
  31. 2023Diversify & Conquer: Outcome-directed Curriculum RL via Out-of-Distribution Disagreementcited by 4
  32. 2024DIDI: Diffusion-Guided Diversity for Offline Behavioral Generationcited by 4
  33. 2022POLTER: Policy Trajectory Ensemble Regularization for Unsupervised Reinforcement Learningcited by 3
  34. 2023Sequential Action-Induced Invariant Representation for Reinforcement Learningcited by 3
  35. 2022Empowered Neural Cellular Automatacited by 2
  36. 2023Self-Supervised Learning of Action Affordances as Interaction Modescited by 2
  37. 2025Learning Skills from Action-Free Videoscited by 2
  38. 2022Build generally reusable agent-environment interaction modelscited by 1

50 most cited of 66 All 66 (JSON)