mlsanityData to 7 Oct 2026

Variational Intrinsic Control

arXiv 1611.07507 ↗First posted 22 Nov 2016cs.LG cs.AIcited by 262

Not on the map. Why?

Abstract (arXiv)

In this paper we introduce a new unsupervised reinforcement learning method for discovering the set of intrinsic options available to an agent. This set is learned by maximizing the number of different states an agent can reliably reach, as measured by the mutual information between the set of options and option termination states. To this end, we instantiate two policy gradient based algorithms, one that creates an explicit embedding space of options and one that represents options implicitly. The algorithms also provide an explicit measure of empowerment in a given state that can be used by an empowerment maximizing agent. The algorithm scales well with function approximation and we demonstrate the applicability of the algorithm on a range of tasks.

Key facts. Cited by 262 papers in the map (data through 7 Oct 2026). First posted on arXiv 22 Nov 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2018: 1; 2019: 0; 2020: 0; 2021: 68; 2022: 53; 2023: 48; 2024: 27; 2025: 38; 2026: 27.

262 papers.

Sort:

By area:

  1. 2018Reward learning from human preferences and demonstrations in Ataricited by 314
  2. 2022Contrastive Learning as Goal-Conditioned Reinforcement Learningcited by 241
  3. 2021How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learnedcited by 201
  4. 2021Behavior From the Void: Unsupervised Active Pre-Trainingcited by 173
  5. 2021URLB: Unsupervised Reinforcement Learning Benchmarkcited by 138
  6. 2021Discovering and Achieving Goals via World Modelscited by 131
  7. 2021APS: Active Pretraining with Successor Featurescited by 109
  8. 2023Motif: Intrinsic Motivation from Artificial Intelligence Feedbackcited by 99
  9. 2022Deep Hierarchical Planning from Pixelscited by 96
  10. 2024Foundation Policies with Hilbert Representationscited by 89
  11. 2021From Motor Control to Team Play in Simulated Humanoid Footballcited by 87
  12. 2023Bootstrap Your Own Skills: Learning to Solve New Tasks with Large Language Model Guidancecited by 85
38 more
  1. 2021Reward is enough for convex MDPscited by 82
  2. 2021Bottom-Up Skill Discovery from Unsegmented Demonstrations for Long-Horizon Robot Manipulationcited by 82
  3. 2023METRA: Scalable Unsupervised RL with Metric-Aware Abstractioncited by 80
  4. 2022Does Zero-Shot Reinforcement Learning Exist?cited by 79
  5. 2023XSkill: Cross Embodiment Skill Discoverycited by 74
  6. 2022CIC: Contrastive Intrinsic Control for Unsupervised Skill Discoverycited by 66
  7. 2023LOTUS: Continual Imitation Learning for Robot Manipulation Through Unsupervised Skill Discoverycited by 63
  8. 2021Learning Task Informed Abstractionscited by 58
  9. 2021Hierarchical Reinforcement Learning By Discovering Intrinsic Optionscited by 57
  10. 2021Exploration in Deep Reinforcement Learning: From Single-Agent to Multiagent Domaincited by 57
  11. 2022Learning Neuro-Symbolic Skills for Bilevel Planningcited by 55
  12. 2021Temporal Abstraction in Reinforcement Learning with the Successor Representationcited by 52
  13. 2025Zero-Shot Whole-Body Humanoid Control via Behavioral Foundation Modelscited by 51
  14. 2021Towards Robust Bisimulation Metric Learningcited by 49
  15. 2023Controllability-Aware Unsupervised Skill Discoverycited by 49
  16. 2023Language Reward Modulation for Pretraining Reinforcement Learningcited by 42
  17. 2021MADE: Exploration via Maximizing Deviation from Explored Regionscited by 40
  18. 2021Modelling Behavioural Diversity for Learning in Open-Ended Gamescited by 39
  19. 2021TRAIL: Near-Optimal Imitation Learning with Suboptimal Datacited by 39
  20. 2021Learning more skills through optimistic explorationcited by 37
  21. 2021Hierarchical Skills for Efficient Explorationcited by 37
  22. 2021The Information Geometry of Unsupervised Reinforcement Learningcited by 36
  23. 2021From Machine Learning to Robotics: Challenges and Opportunities for Embodied Intelligencecited by 35
  24. 2022Discovering Policies with DOMiNO: Diversity Optimization Maintaining Near Optimalitycited by 35
  25. 2023Contrastive Difference Predictive Codingcited by 35
  26. 2021Autonomous Reinforcement Learning: Formalism and Benchmarkingcited by 33
  27. 2021Geometric Entropic Explorationcited by 32
  28. 2022Choreographer: Learning and Adapting Skills in Imaginationcited by 31
  29. 2022Challenging Common Assumptions in Convex Reinforcement Learningcited by 29
  30. 2021Which Mutual-Information Representation Learning Objectives are Sufficient for Control?cited by 28
  31. 2021Unsupervised Skill Discovery with Bottleneck Option Learningcited by 28
  32. 2021Provable RL with Exogenous Distractors via Multistep Inverse Dynamicscited by 28
  33. 2023Deep Laplacian-based Options for Temporally-Extended Explorationcited by 28
  34. 2021Discovery of Options via Meta-Learned Subgoalscited by 27
  35. 2022INFOrmation Prioritization through EmPOWERment in Visual Model-Based RLcited by 27
  36. 2021Learning Transferable Motor Skills with Hierarchical Latent Mixture Policiescited by 26
  37. 2023Behavior Contrastive Learning for Unsupervised Skill Discoverycited by 26
  38. 2022An information-theoretic perspective on intrinsic motivation in reinforcement learning: a surveycited by 25

50 most cited of 262 All 262 (JSON)