# Relative Variational Intrinsic Control

arXiv: https://arxiv.org/abs/2012.07827 · HTML: https://mlsanity.com/paper/2012.07827/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Dec 2020.

## Built on by (most cited first, 30 of 34)

- [METRA: Scalable Unsupervised RL with Metric-Aware Abstraction](https://mlsanity.com/paper/2310.08887/) (2023)
- [Learning more skills through optimistic exploration](https://mlsanity.com/paper/2107.14226/) (2021)
- [Discovering Policies with DOMiNO: Diversity Optimization Maintaining Near Optimality](https://mlsanity.com/paper/2205.13521/) (2022)
- [An information-theoretic perspective on intrinsic motivation in reinforcement learning: a survey](https://mlsanity.com/paper/2209.08890/) (2022)
- [Discovery of Options via Meta-Learned Subgoals](https://mlsanity.com/paper/2102.06741/) (2021)
- [Behavior Contrastive Learning for Unsupervised Skill Discovery](https://mlsanity.com/paper/2305.04477/) (2023)
- [Wasserstein Unsupervised Reinforcement Learning](https://mlsanity.com/paper/2110.07940/) (2021)
- [Versatile Skill Control via Self-supervised Adversarial Imitation of Unlabeled Mixed Motions](https://mlsanity.com/paper/2209.07899/) (2022)
- [Can a MISL Fly? Analysis and Ingredients for Mutual Information Skill Learning](https://mlsanity.com/paper/2412.08021/) (2024)
- [Discovering Diverse Nearly Optimal Policies with Successor Features](https://mlsanity.com/paper/2106.00669/) (2021)
- [Direct then Diffuse: Incremental Unsupervised Skill Discovery for State Covering and Goal Reaching](https://mlsanity.com/paper/2110.14457/) (2021)
- [Pretraining in Deep Reinforcement Learning: A Survey](https://mlsanity.com/paper/2211.03959/) (2022)
- [Diversifying AI: Towards Creative Chess with AlphaZero](https://mlsanity.com/paper/2308.09175/) (2023)
- [Plasticity as the Mirror of Empowerment](https://mlsanity.com/paper/2505.10361/) (2025)
- [Understanding the Origin of Information-Seeking Exploration in Probabilistic Objectives for Control](https://mlsanity.com/paper/2103.06859/) (2021)
- [A Mixture of Surprises for Unsupervised Reinforcement Learning](https://mlsanity.com/paper/2210.06702/) (2022)
- [Disentangled Unsupervised Skill Discovery for Efficient Hierarchical Reinforcement Learning](https://mlsanity.com/paper/2410.11251/) (2024)
- [Intrinsic Motivation in Dynamical Control Systems](https://mlsanity.com/paper/2301.00005/) (2022)
- [Adapt On-the-Go: Behavior Modulation for Single-Life Robot Deployment](https://mlsanity.com/paper/2311.01059/) (2023)
- [A Unified Algorithm Framework for Unsupervised Discovery of Skills based on Determinantal Point Process](https://mlsanity.com/paper/2212.00211/) (2022)
- [Hierarchical Empowerment: Towards Tractable Empowerment-Based Skill Learning](https://mlsanity.com/paper/2307.02728/) (2023)
- [Wasserstein Distance Maximizing Intrinsic Control](https://mlsanity.com/paper/2110.15331/) (2021)
- [ASE: Large-Scale Reusable Adversarial Skill Embeddings for Physically Simulated Characters](https://mlsanity.com/paper/2205.01906/) (2022)
- [Skill-Based Reinforcement Learning with Intrinsic Reward Matching](https://mlsanity.com/paper/2210.07426/) (2022)
- [Curiosity in Hindsight: Intrinsic Exploration in Stochastic Environments](https://mlsanity.com/paper/2211.10515/) (2022)
- [POLTER: Policy Trajectory Ensemble Regularization for Unsupervised Reinforcement Learning](https://mlsanity.com/paper/2205.11357/) (2022)
- [Beyond Rewards: a Hierarchical Perspective on Offline Multiagent Behavioral Analysis](https://mlsanity.com/paper/2206.09046/) (2022)
- [Learning Temporally Extended Skills in Continuous Domains as Symbolic Actions for Planning](https://mlsanity.com/paper/2207.05018/) (2022)
- [Diversity Progress for Goal Selection in Discriminability-Motivated RL](https://mlsanity.com/paper/2411.01521/) (2024)
- [VendiRL: A Framework for Self-Supervised Reinforcement Learning of Diversely Diverse Skills](https://mlsanity.com/paper/2509.02930/) (2025)

## Abstract (arXiv)

In the absence of external rewards, agents can still learn useful behaviors by identifying and mastering a set of diverse skills within their environment. Existing skill learning methods use mutual information objectives to incentivize each skill to be diverse and distinguishable from the rest. However, if care is not taken to constrain the ways in which the skills are diverse, trivially diverse skill sets can arise. To ensure useful skill diversity, we propose a novel skill learning objective, Relative Variational Intrinsic Control (RVIC), which incentivizes learning skills that are distinguishable in how they change the agent's relationship to its environment. The resulting set of skills tiles the space of affordances available to the agent. We qualitatively analyze skill behaviors on multiple environments and show how RVIC skills are more useful than skills discovered by existing methods when used in hierarchical reinforcement learning.