mlsanityData to 7 Oct 2026

Wasserstein Distance Maximizing Intrinsic Control

arXiv 2110.15331 ↗First posted 28 Oct 2021cs.LG cs.AIcited by 5

Not on the map. Why?

Abstract (arXiv)

This paper deals with the problem of learning a skill-conditioned policy that acts meaningfully in the absence of a reward signal. Mutual information based objectives have shown some success in learning skills that reach a diverse set of states in this setting. These objectives include a KL-divergence term, which is maximized by visiting distinct states even if those states are not far apart in the MDP. This paper presents an approach that rewards the agent for learning skills that maximize the Wasserstein distance of their state visitation from the start state of the skill. It shows that such an objective leads to a policy that covers more distance in the MDP than diversity based objectives, and validates the results on a variety of Atari environments.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 18 of its 41 references that are in the map. First posted on arXiv 28 Oct 2021.

Builds on

18 of 41 references.

References · 18

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2017Improved Training of Wasserstein GANscited by 2,223
  3. 2018IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architecturescited by 866
  4. 2018Computational Optimal Transportcited by 765
  5. 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 734
  6. 2016Unifying Count-Based Exploration and Intrinsic Motivationcited by 632
  7. 2017Revisiting the Arcade Learning Environment: Evaluation Protocols and Open Problems for General Agentscited by 267
  8. 2016Variational Intrinsic Controlcited by 262
  9. 2020Explore, Discover and Learn: Unsupervised Discovery of State-Covering Skillscited by 120
  10. 2020Primal Wasserstein Imitation Learningcited by 120
  11. 2018On Learning Intrinsic Rewards for Policy Gradient Methodscited by 100
  12. 2017Intrinsically Motivated Goal Exploration Processes with Automatic Curriculum Learningcited by 79
6 more
  1. 2018Synthesizing Programs for Images using Reinforced Adversarial Learningcited by 72
  2. 2019Wasserstein Adversarial Imitation Learningcited by 59
  3. 2021Adversarial Intrinsic Motivation for Reinforcement Learningcited by 40
  4. 2019What Can Learned Intrinsic Rewards Capture?cited by 36
  5. 2020Relative Variational Intrinsic Controlcited by 34
  6. 2020Wasserstein Distance guided Adversarial Imitation Learning with Reward Shape Explorationcited by 13

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 2; 2024: 0; 2025: 2.

5 papers.

  1. 2022Neuroevolution is a Competitive Alternative to Reinforcement Learning for Skill Discoverycited by 23
  2. 2023Outcome-directed Reinforcement Learning by Uncertainty & Temporal Distance-Aware Curriculum Goal Generationcited by 14
  3. 2023CQM: Curriculum Reinforcement Learning with a Quantized World Modelcited by 11
  4. 2025Task Adaptation from Skills: Information Geometry, Disentanglement, and New Objectives for Unsupervised Reinforcement Learningcited by 5
  5. 2025Imagine, Verify, Execute: Memory-guided Agentic Exploration with Vision-Language Modelscited by 2