# Variational Intrinsic Control Revisited

arXiv: https://arxiv.org/abs/2010.03281 · HTML: https://mlsanity.com/paper/2010.03281/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Oct 2020.

## Built on by (most cited first, 11 of 11)

- [CIC: Contrastive Intrinsic Control for Unsupervised Skill Discovery](https://mlsanity.com/paper/2202.00161/) (2022)
- [Wasserstein Unsupervised Reinforcement Learning](https://mlsanity.com/paper/2110.07940/) (2021)
- [Language Guided Skill Discovery](https://mlsanity.com/paper/2406.06615/) (2024)
- [LECO: Learnable Episodic Count for Task-Specific Intrinsic Reward](https://mlsanity.com/paper/2210.05409/) (2022)
- [Intrinsic Motivation in Dynamical Control Systems](https://mlsanity.com/paper/2301.00005/) (2022)
- [Meta-Learning Parameterized Skills](https://mlsanity.com/paper/2206.03597/) (2022)
- [A Unified Algorithm Framework for Unsupervised Discovery of Skills based on Determinantal Point Process](https://mlsanity.com/paper/2212.00211/) (2022)
- [Curiosity in Hindsight: Intrinsic Exploration in Stochastic Environments](https://mlsanity.com/paper/2211.10515/) (2022)
- [SuPLE: Robot Learning with Lyapunov Rewards](https://mlsanity.com/paper/2411.13613/) (2024)
- [APART: Diverse Skill Discovery using All Pairs with Ascending Reward and DropouT](https://mlsanity.com/paper/2308.12649/) (2023)
- [Game-Guided Skill Discovery through Self-Play for Playable Agent Control](https://mlsanity.com/paper/2609.40137/) (2026)

## Abstract (arXiv)

In this paper, we revisit variational intrinsic control (VIC), an unsupervised reinforcement learning method for finding the largest set of intrinsic options available to an agent. In the original work by Gregor et al. (2016), two VIC algorithms were proposed: one that represents the options explicitly, and the other that does it implicitly. We show that the intrinsic reward used in the latter is subject to bias in stochastic environments, causing convergence to suboptimal solutions. To correct this behavior and achieve the maximal empowerment, we propose two methods respectively based on the transitional probability model and Gaussian mixture model. We substantiate our claims through rigorous mathematical derivations and experimental analyses.