mlsanityData to 7 Oct 2026

Learning with AMIGo: Adversarially Motivated Intrinsic Goals

arXiv 2006.12122 ↗First posted 22 Jun 2020cs.LG cs.AI stat.MLcited by 86

Not on the map. Why?

Abstract (arXiv)

A key challenge for reinforcement learning (RL) consists of learning in environments with sparse extrinsic rewards. In contrast to current RL methods, humans are able to learn new skills with little or no reward by using various forms of intrinsic motivation. We propose AMIGo, a novel agent incorporating -- as form of meta-learning -- a goal-generating teacher that proposes Adversarially Motivated Intrinsic Goals to train a goal-conditioned "student" policy in the absence of (or alongside) environment reward. Specifically, through a simple but effective "constructively adversarial" objective, the teacher learns to propose increasingly challenging -- yet achievable -- goals that allow the student to learn general skills for acting in a new environment, independent of the task to be solved. We show that our method generates a natural curriculum of self-proposed goals which ultimately allows the agent to solve challenging procedurally-generated tasks where other forms of intrinsic motivation and state-of-the-art RL methods fail.

Key facts. Cited by 86 papers in the map (data through 7 Oct 2026). First posted on arXiv 22 Jun 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 17; 2022: 32; 2023: 13; 2024: 13; 2025: 6; 2026: 5.

86 papers.

Sort:

By area:

  1. 2021Open-Ended Learning Leads to Generally Capable Agentscited by 177
  2. 2022Evolving Curricula with Regret-Based Environment Designcited by 166
  3. 2022Goal-Conditioned Reinforcement Learning: Problems and Solutionscited by 145
  4. 2023Human-Timescale Adaptation in an Open-Ended Task Spacecited by 136
  5. 2021Replay-Guided Adversarial Environment Designcited by 108
  6. 2022Automated Reinforcement Learning (AutoRL): A Survey and Open Problemscited by 69
  7. 2023OMNI: Open-endedness via Models of human Notions of Interestingnesscited by 66
  8. 2022Semantic Exploration from Language Abstractions and Pretrained Representationscited by 51
  9. 2022Exploration via Elliptical Episodic Bonusescited by 51
  10. 2022Improving Intrinsic Exploration with Language Abstractionscited by 49
  11. 2021Adversarially Guided Actor-Criticcited by 47
  12. 2024Learning Formal Mathematics From Intrinsic Motivationcited by 45
38 more
  1. 2021ELLA: Exploration through Learned Language Abstractioncited by 42
  2. 2021MADE: Exploration via Maximizing Deviation from Explored Regionscited by 40
  3. 2021Learning General Optimal Policies with Graph Neural Networks: Expressive Power, Transparency, and Limitscited by 38
  4. 2021Multi-task curriculum learning in a complex, visual, hard-exploration domain: Minecraftcited by 34
  5. 2022Environment Generation for Zero-Shot Compositional Reinforcement Learningcited by 32
  6. 2022LEAGUE: Guided Skill Learning and Abstraction for Long-Horizon Manipulationcited by 32
  7. 2021Rank the Episodes: A Simple Approach for Exploration in Procedurally-Generated Environmentscited by 28
  8. 2021Variational Automatic Curriculum Learning for Sparse-Reward Cooperative Multi-Agent Problemscited by 25
  9. 2023Learning Goal-Conditioned Policies Offline with Self-Supervised Reward Shapingcited by 25
  10. 2022Towards Deployment-Efficient Reinforcement Learning: Lower Bound and Optimalitycited by 24
  11. 2022Learning Generalized Policies Without Supervision Using GNNscited by 22
  12. 2024A Single Goal is All You Need: Skills and Exploration Emerge from Contrastive RL without Rewards, Demonstrations, or Subgoalscited by 20
  13. 2023A Study of Global and Episodic Bonuses for Exploration in Contextual MDPscited by 19
  14. 2021TeachMyAgent: a Benchmark for Automatic Curriculum Learning in Deep RLcited by 18
  15. 2022Grounding Aleatoric Uncertainty for Unsupervised Environment Designcited by 16
  16. 2022It Takes Four to Tango: Multiagent Selfplay for Automatic Curriculum Generationcited by 14
  17. 2024Solving Minimum-Cost Reach Avoid using Reinforcement Learningcited by 14
  18. 2021Adversarial Environment Generation for Learning to Navigate the Webcited by 13
  19. 2022Learning General World Models in a Handful of Reward-Free Deploymentscited by 12
  20. 2022LECO: Learnable Episodic Count for Task-Specific Intrinsic Rewardcited by 11
  21. 2022Perceiving the World: Question-guided Reinforcement Learning for Text-based Gamescited by 10
  22. 2022Discrete Factorial Representations as an Abstraction for Goal Conditioned Reinforcement Learningcited by 10
  23. 2022Exploring through Random Curiosity with General Value Functionscited by 10
  24. 2021Situated Dialogue Learning through Procedural Environment Generationcited by 9
  25. 2023minimax: Efficient Baselines for Autocurricula in JAXcited by 9
  26. 2021Learning Continuous Environment Fields via Implicit Functionscited by 8
  27. 2022State-Conditioned Adversarial Subgoal Generationcited by 8
  28. 2022An Evaluation Study of Intrinsic Motivation Techniques applied to Reinforcement Learning over Hard Exploration Environmentscited by 8
  29. 2021Don't Do What Doesn't Matter: Intrinsic Motivation with Action Usefulnesscited by 7
  30. 2021Explore and Control with Adversarial Surprisecited by 6
  31. 2022From Psychological Curiosity to Artificial Curiosity: Curiosity-Driven Learning in Artificial Intelligence Taskscited by 6
  32. 2023Building Open-Ended Embodied Agent via Language-Policy Bidirectional Adaptationcited by 6
  33. 2024Learning World Models for Unconstrained Goal Navigationcited by 6
  34. 2022Open-Ended Reinforcement Learning with Neural Reward Functionscited by 5
  35. 2025Hierarchical Reinforcement Learning with Uncertainty-Guided Diffusional Subgoalscited by 5
  36. 2022Feasible Adversarial Robust Reinforcement Learning for Underspecified Environmentscited by 4
  37. 2023Robust Subtask Learning for Compositional Generalizationcited by 4
  38. 2026Recursive Agent Optimizationcited by 4

50 most cited of 86 All 86 (JSON)