mlsanityData to 14 Oct 2026

Intrinsic Motivation and Automatic Curricula via Asymmetric Self-Play

arXiv 1703.05407 ↗First posted 15 Mar 2017cs.LGcited by 141

Not on the map. Why?

Abstract (arXiv)

We describe a simple scheme that allows an agent to learn about its environment in an unsupervised manner. Our scheme pits two versions of the same agent, Alice and Bob, against one another. Alice proposes a task for Bob to complete; and then Bob attempts to complete the task. In this work we will focus on two kinds of environments: (nearly) reversible environments and environments that can be reset. Alice will "propose" the task by doing a sequence of actions and then Bob must undo or repeat them, respectively. Via an appropriate reward structure, Alice and Bob automatically generate a curriculum of exploration, enabling unsupervised training of the agent. When Bob is deployed on an RL task within the environment, this unsupervised training reduces the number of supervised episodes needed to learn, and in some cases converges to a higher reward.

Key facts. Cited by 141 papers in the map (data through 14 Oct 2026). First posted on arXiv 15 Mar 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 31; 2022: 21; 2023: 26; 2024: 19; 2025: 24; 2026: 20.

141 papers.

Sort:

By area:

  1. 2025Absolute Zero: Reinforced Self-play Reasoning with Zero Datacited by 344
  2. 2021Open-Ended Learning Leads to Generally Capable Agentscited by 182
  3. 2022Evolving Curricula with Regret-Based Environment Designcited by 173
  4. 2023Human-Timescale Adaptation in an Open-Ended Task Spacecited by 138
  5. 2021Discovering and Achieving Goals via World Modelscited by 131
  6. 2021Replay-Guided Adversarial Environment Designcited by 111
  7. 2025SPIRAL: Self-Play on Zero-Sum Games Incentivizes Reasoning via Multi-Agent Multi-Turn Reinforcement Learningcited by 100
  8. 2021Scalable Evaluation of Multi-Agent Reinforcement Learning with Melting Potcited by 99
  9. 2021Reset-Free Reinforcement Learning via Multi-Task Learning: Learning Dexterous Manipulation Behaviors without Human Interventioncited by 95
  10. 2025Efficient Reinforcement Finetuning via Adaptive Curriculum Learningcited by 93
  11. 2024Craftax: A Lightning-Fast Benchmark for Open-Ended Reinforcement Learningcited by 88
  12. 2023METRA: Scalable Unsupervised RL with Metric-Aware Abstractioncited by 86
38 more
  1. 2025SPICE: Self-Play In Corpus Environments Improves Reasoningcited by 83
  2. 2021Asymmetric self-play for automatic goal discovery in robotic manipulationcited by 72
  3. 2022Automated Reinforcement Learning (AutoRL): A Survey and Open Problemscited by 70
  4. 2025Self-Challenging Language Model Agentscited by 59
  5. 2021Exploration in Deep Reinforcement Learning: From Single-Agent to Multiagent Domaincited by 58
  6. 2022Exploration via Elliptical Episodic Bonusescited by 51
  7. 2025Self-Questioning Language Modelscited by 50
  8. 2023Controllability-Aware Unsupervised Skill Discoverycited by 49
  9. 2023Towards A Unified Agent with Foundation Modelscited by 47
  10. 2025Guided Self-Evolving LLMs with Minimal Human Supervisioncited by 35
  11. 2022LEAGUE: Guided Skill Learning and Abstraction for Long-Horizon Manipulationcited by 34
  12. 2021Autonomous Reinforcement Learning via Subgoal Curriculacited by 33
  13. 2022Environment Generation for Zero-Shot Compositional Reinforcement Learningcited by 32
  14. 2021Towards Distraction-Robust Active Visual Trackingcited by 30
  15. 2021Rank the Episodes: A Simple Approach for Exploration in Procedurally-Generated Environmentscited by 28
  16. 2021Unsupervised Skill Discovery with Bottleneck Option Learningcited by 28
  17. 2021Variational Automatic Curriculum Learning for Sparse-Reward Cooperative Multi-Agent Problemscited by 26
  18. 2021Self-Paced Context Evaluation for Contextual Reinforcement Learningcited by 25
  19. 2023Learning Goal-Conditioned Policies Offline with Self-Supervised Reward Shapingcited by 25
  20. 2024A Single Goal is All You Need: Skills and Exploration Emerge from Contrastive RL without Rewards, Demonstrations, or Subgoalscited by 23
  21. 2024Beyond Sparse Rewards: Enhancing Reinforcement Learning with Language Model Critique in Text Generationcited by 22
  22. 2024Scalable Reinforcement Post-Training Beyond Static Human Prompts: Evolving Alignment via Asymmetric Self-Playcited by 21
  23. 2023Proximal Curriculum for Reinforcement Learning Agentscited by 20
  24. 2024A social path to human-like artificial intelligencecited by 20
  25. 2024Refining Minimax Regret for Unsupervised Environment Designcited by 18
  26. 2021Learn Goal-Conditioned Policy with Intrinsic Motivation for Deep Reinforcement Learningcited by 17
  27. 2023FastRLAP: A System for Learning High-Speed Driving via Deep RL and Autonomous Practicingcited by 17
  28. 2023Stabilizing Unsupervised Environment Design with a Learned Adversarycited by 17
  29. 2024MU-Bench: A Multitask Multimodal Benchmark for Machine Unlearningcited by 17
  30. 2022Grounding Aleatoric Uncertainty for Unsupervised Environment Designcited by 16
  31. 2025RL for Reasoning by Adaptively Revealing Rationalescited by 16
  32. 2023Outcome-directed Reinforcement Learning by Uncertainty & Temporal Distance-Aware Curriculum Goal Generationcited by 15
  33. 2021Is Curiosity All You Need? On the Utility of Emergent Behaviours from Curious Explorationcited by 14
  34. 2022It Takes Four to Tango: Multiagent Selfplay for Automatic Curriculum Generationcited by 14
  35. 2024Acquiring Diverse Skills using Curriculum Reinforcement Learning with Mixture of Expertscited by 14
  36. 2024Can Learned Optimization Make Reinforcement Learning Less Difficult?cited by 14
  37. 2025Normalizing Flows are Capable Models for Continuous Controlcited by 14
  38. 2021Adversarial Environment Generation for Learning to Navigate the Webcited by 13

50 most cited of 141 All 141 (JSON)