mlsanityData to 13 Oct 2026

Unsupervised Control Through Non-Parametric Discriminative Rewards

arXiv 1811.11359 ↗First posted 28 Nov 2018cs.LG cs.AI stat.MLcited by 102

Not on the map. Why?

Abstract (arXiv)

Learning to control an environment without hand-crafted rewards or expert data remains challenging and is at the frontier of reinforcement learning research. We present an unsupervised learning algorithm to train agents to achieve perceptually-specified goals using only a stream of observations and actions. Our agent simultaneously learns a goal-conditioned policy and a goal achievement reward function that measures how similar a state is to the goal state. This dual optimization leads to a co-operative game, giving rise to a learned reward function that reflects similarity in controllable aspects of the environment instead of distance in the space of observations. We demonstrate the efficacy of our agent to learn, in an unsupervised manner, to reach a diverse set of goals on three domains -- Atari, the DeepMind Control Suite and DeepMind Lab.

Key facts. Cited by 102 papers in the map (data through 13 Oct 2026). First posted on arXiv 28 Nov 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2021: 31; 2022: 25; 2023: 20; 2024: 10; 2025: 10; 2026: 6.

102 papers.

Sort:

By area:

  1. 2022Contrastive Learning as Goal-Conditioned Reinforcement Learningcited by 244
  2. 2021Behavior From the Void: Unsupervised Active Pre-Trainingcited by 173
  3. 2022Goal-Conditioned Reinforcement Learning: Problems and Solutionscited by 148
  4. 2021Discovering and Achieving Goals via World Modelscited by 131
  5. 2021APS: Active Pretraining with Successor Featurescited by 109
  6. 2022Deep Hierarchical Planning from Pixelscited by 99
  7. 2023Bootstrap Your Own Skills: Learning to Solve New Tasks with Large Language Model Guidancecited by 85
  8. 2023METRA: Scalable Unsupervised RL with Metric-Aware Abstractioncited by 82
  9. 2022BYOL-Explore: Exploration by Bootstrapped Predictioncited by 71
  10. 2022CIC: Contrastive Intrinsic Control for Unsupervised Skill Discoverycited by 66
  11. 2021Hierarchical Reinforcement Learning By Discovering Intrinsic Optionscited by 57
  12. 2021Offline Meta-Reinforcement Learning with Online Self-Supervisioncited by 52
38 more
  1. 2025Zero-Shot Whole-Body Humanoid Control via Behavioral Foundation Modelscited by 52
  2. 2023Controllability-Aware Unsupervised Skill Discoverycited by 49
  3. 2021TRAIL: Near-Optimal Imitation Learning with Suboptimal Datacited by 39
  4. 2022Bisimulation Makes Analogies in Goal-Conditioned Reinforcement Learningcited by 38
  5. 2021Learning more skills through optimistic explorationcited by 37
  6. 2021The Information Geometry of Unsupervised Reinforcement Learningcited by 37
  7. 2023Contrastive Difference Predictive Codingcited by 35
  8. 2023Augmenting Autotelic Agents with Large Language Modelscited by 29
  9. 2022An information-theoretic perspective on intrinsic motivation in reinforcement learning: a surveycited by 27
  10. 2021Value Function Spaces: Skill-Centric State Abstractions for Long-Horizon Reasoningcited by 25
  11. 2023Learning Goal-Conditioned Policies Offline with Self-Supervised Reward Shapingcited by 25
  12. 2021MURAL: Meta-Learning Uncertainty-Aware Rewards for Outcome-Driven Reinforcement Learningcited by 24
  13. 2022The Free Energy Principle for Perception and Action: A Deep Learning Perspectivecited by 24
  14. 2023ALAN: Autonomously Exploring Robotic Agents in the Real Worldcited by 23
  15. 2021What Can I Do Here? Learning New Skills by Imagining Visual Affordancescited by 22
  16. 2021Contrastive Active Inferencecited by 21
  17. 2024Can a MISL Fly? Analysis and Ingredients for Mutual Information Skill Learningcited by 20
  18. 2021Solving Compositional Reinforcement Learning Problems via Task Reductioncited by 19
  19. 2021Direct then Diffuse: Incremental Unsupervised Skill Discovery for State Covering and Goal Reachingcited by 19
  20. 2021Mutual Information State Intrinsic Controlcited by 18
  21. 2022Pretraining in Deep Reinforcement Learning: A Surveycited by 18
  22. 2024Proto Successor Measure: Representing the Behavior Space of an RL Agentcited by 18
  23. 2021Outcome-Driven Reinforcement Learning via Variational Inferencecited by 17
  24. 2021Active Hierarchical Exploration with Stable Subgoal Representation Learningcited by 17
  25. 2021Understanding the World Through Actioncited by 17
  26. 2022Understanding the Complexity Gains of Single-Task RL with a Curriculumcited by 17
  27. 2021Policy Information Capacity: Information-Theoretic Measure for Task Complexity in Deep Reinforcement Learningcited by 15
  28. 2021Learn Goal-Conditioned Policy with Intrinsic Motivation for Deep Reinforcement Learningcited by 15
  29. 2021Variational Empowerment as Representation Learning for Goal-Based Reinforcement Learningcited by 15
  30. 2021Unsupervised Domain Adaptation with Dynamics-Aware Rewards in Reinforcement Learningcited by 15
  31. 2023Outcome-directed Reinforcement Learning by Uncertainty & Temporal Distance-Aware Curriculum Goal Generationcited by 15
  32. 2021SimSR: Simple Distance-based State Representation for Deep Reinforcement Learningcited by 14
  33. 2024Solving Minimum-Cost Reach Avoid using Reinforcement Learningcited by 14
  34. 2022Learning Robotic Navigation from Experience: Principles, Methods, and Recent Resultscited by 13
  35. 2024EXTRACT: Efficient Policy Learning by Extracting Transferable Robot Skills from Offline Datacited by 13
  36. 2025DISCOVER: Automated Curricula for Sparse-Reward Reinforcement Learningcited by 12
  37. 2021Braxlines: Fast and Interactive Toolkit for RL-driven Behavior Engineering beyond Reward Maximizationcited by 11
  38. 2022Learning on the Job: Self-Rewarding Offline-to-Online Finetuning for Industrial Insertion of Novel Connectors from Visioncited by 11

50 most cited of 102 All 102 (JSON)