mlsanityData to 7 Oct 2026

Mutual Information State Intrinsic Control

arXiv 2103.08107 ↗First posted 15 Mar 2021cs.LGcited by 18

Not on the map. Why?

Abstract (arXiv)

Reinforcement learning has been shown to be highly successful at many challenging tasks. However, success heavily relies on well-shaped rewards. Intrinsically motivated RL attempts to remove this constraint by defining an intrinsic reward function. Motivated by the self-consciousness concept in psychology, we make a natural assumption that the agent knows what constitutes itself, and propose a new intrinsic objective that encourages the agent to have maximum control on the environment. We mathematically formalize this reward as the mutual information between the agent state and the surrounding state under the current agent policy. With this new intrinsic motivation, we are able to outperform previous methods, including being able to complete the pick-and-place task for the first time without using any task reward. A video showing experimental results is available at https://youtu.be/AUCwc9RThpk.

Key facts. Cited by 18 papers in the map (data through 7 Oct 2026). Builds on 26 of its 41 references that are in the map. First posted on arXiv 15 Mar 2021.

Builds on

26 of 41 references.

References · 26

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  4. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  5. 2015Continuous control with deep reinforcement learningcited by 3,279
  6. 2018Learning deep representations by mutual information estimation and maximizationcited by 1,265
  7. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,173
  8. 2018Deep Graph Infomaxcited by 1,101
  9. 2017Hindsight Experience Replaycited by 1,096
  10. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,084
  11. 2015Prioritized Experience Replaycited by 1,070
  12. 2016Deep Variational Information Bottleneckcited by 915
14 more
  1. 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 734
  2. 2018Large-Scale Study of Curiosity-Driven Learningcited by 277
  3. 2016Variational Intrinsic Controlcited by 262
  4. 2018Multi-Goal Reinforcement Learning: Challenging Robotics Environments and Request for Researchcited by 256
  5. 2019Skew-Fit: State-Covering Self-Supervised Reinforcement Learningcited by 171
  6. 2015Variational Information Maximisation for Intrinsically Motivated Reinforcement Learningcited by 142
  7. 2020Explore, Discover and Learn: Unsupervised Discovery of State-Covering Skillscited by 120
  8. 2018Unsupervised Control Through Non-Parametric Discriminative Rewardscited by 102
  9. 2017Nonlinear Information Bottleneckcited by 71
  10. 2013Empowerment -- an Introductioncited by 65
  11. 2019Maximum Entropy-Regularized Multi-Goal Reinforcement Learningcited by 39
  12. 2016Relevant sparse codes with variational information bottleneckcited by 17
  13. 2017Feature Control as Intrinsic Motivation for Hierarchical Reinforcement Learningcited by 15
  14. 2019Mega-Reward: Achieving Human-Level Play without Extrinsic Rewardscited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 4; 2022: 3; 2023: 6; 2024: 2; 2025: 2; 2026: 1.

18 papers.

Sort:

  1. 2021Causal Influence Detection for Improving Efficiency in Reinforcement Learningcited by 70
  2. 2021Maximum Entropy Population-Based Training for Zero-Shot Human-AI Coordinationcited by 69
  3. 2022Lipschitz-constrained Unsupervised Skill Discoverycited by 60
  4. 2023Controllability-Aware Unsupervised Skill Discoverycited by 49
  5. 2023Behavior Contrastive Learning for Unsupervised Skill Discoverycited by 26
  6. 2022An information-theoretic perspective on intrinsic motivation in reinforcement learning: a surveycited by 25
  7. 2021Self-supervised Reinforcement Learning with Independently Controllable Subgoalscited by 16
  8. 2023Pre-Training and Fine-Tuning Generative Flow Networkscited by 16
  9. 2022Policy-Based Bayesian Experimental Design for Non-Differentiable Implicit Modelscited by 15
  10. 2023Outcome-directed Reinforcement Learning by Uncertainty & Temporal Distance-Aware Curriculum Goal Generationcited by 14
  11. 2021Braxlines: Fast and Interactive Toolkit for RL-driven Behavior Engineering beyond Reward Maximizationcited by 11
  12. 2024Disentangled Unsupervised Skill Discovery for Efficient Hierarchical Reinforcement Learningcited by 9
6 more
  1. 2023ELDEN: Exploration via Local Dependenciescited by 8
  2. 2023Diversify & Conquer: Outcome-directed Curriculum RL via Out-of-Distribution Disagreementcited by 4
  3. 2024SLIM: Skill Learning with Multiple Criticscited by 3
  4. 2026Behavioral Mode Discovery for Fine-tuning Multimodal Generative Policiescited by 2
  5. 2025CAIMAN: Causal Action Influence Detection for Sample-efficient Loco-manipulationcited by 0
  6. 2025AMPED: Adaptive Multi-objective Projection for balancing Exploration and skill Diversificationcited by 0