mlsanityData to 14 Oct 2026

Multi-task curriculum learning in a complex, visual, hard-exploration domain: Minecraft

arXiv 2106.14876 ↗First posted 28 Jun 2021cs.LG stat.MLcited by 35

Not on the map. Why?

Abstract (arXiv)

An important challenge in reinforcement learning is training agents that can solve a wide variety of tasks. If tasks depend on each other (e.g. needing to learn to walk before learning to run), curriculum learning can speed up learning by focusing on the next best task to learn. We explore curriculum learning in a complex, visual domain with many hard exploration challenges: Minecraft. We find that learning progress (defined as a change in success probability of a task) is a reliable measure of learnability for automatically constructing an effective curriculum. We introduce a learning-progress based curriculum and test it on a complex reinforcement learning problem (called "Simon Says") where an agent is instructed to obtain a desired goal item. Many of the required skills depend on each other. Experiments demonstrate that: (1) a within-episode exploration bonus for obtaining new items improves performance, (2) dynamically adjusting this bonus across training such that it only applies to items the agent cannot reliably obtain yet further increases performance, (3) the learning-progress based curriculum elegantly follows the learning curve of the agent, and (4) when the learning-progress based curriculum is combined with the dynamic exploration bonus it learns much more efficiently and obtains far higher performance than uniform baselines. These results suggest that combining intra-episode and across-training exploration bonuses with learning progress creates a promising method for automated curriculum generation, which may substantially increase our ability to train more capable, generally intelligent agents.

Key facts. Cited by 35 papers in the map (data through 14 Oct 2026). Builds on 22 of its 29 references that are in the map. First posted on arXiv 28 Jun 2021.

Builds on

22 of 29 references.

References · 22

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 2,026
  3. 2018IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architecturescited by 883
  4. 2019Solving Rubik's Cube with a Robot Handcited by 734
  5. 2016Unifying Count-Based Exploration and Intrinsic Motivationcited by 639
  6. 2020Emergent Complexity and Zero-shot Transfer via Unsupervised Environment Designcited by 275
  7. 2020First return, then explorecited by 266
  8. 2017Automatic Goal Generation for Reinforcement Learning Agentscited by 259
  9. 2017Automated Curriculum Learning for Neural Networkscited by 257
  10. 2019MineRL: A Large-Scale Dataset of Minecraft Demonstrationscited by 215
  11. 2019Go-Explore: a New Approach for Hard-Exploration Problemscited by 212
  12. 2017Teacher-Student Curriculum Learningcited by 210
10 more
  1. 2019Paired Open-Ended Trailblazer (POET): Endlessly Generating Increasingly Complex and Diverse Learning Environments and Their Solutionscited by 192
  2. 2019Teacher algorithms for curriculum learning of Deep RL in continuously parameterized environmentscited by 119
  3. 2020Enhanced POET: Open-Ended Reinforcement Learning through Unbounded Invention of Learning Challenges and their Solutionscited by 106
  4. 2020Learning with AMIGo: Adversarially Motivated Intrinsic Goalscited by 88
  5. 2020Automatic Curriculum Learning through Value Disagreementcited by 87
  6. 2021Asymmetric self-play for automatic goal discovery in robotic manipulationcited by 72
  7. 2020Active World Model Learning with Progress Curiositycited by 28
  8. 2018Learning Goal Embeddings via Self-Play for Hierarchical Reinforcement Learningcited by 24
  9. 2021Adversarial Environment Generation for Learning to Navigate the Webcited by 13
  10. 2018Deep Curiosity Search: Intra-Life Exploration Can Improve Performance on Challenging Deep Reinforcement Learning Problemscited by 12

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 1; 2022: 4; 2023: 11; 2024: 5; 2025: 7; 2026: 7.

35 papers.

Sort:

By area:

  1. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,191
  2. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,439
  3. 2023Mastering Diverse Domains through World Modelscited by 1,253
  4. 2022MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledgecited by 478
  5. 2025Darwin Godel Machine: Open-Ended Evolution of Self-Improving Agentscited by 356
  6. 2022Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videoscited by 345
  7. 2023Ghost in the Minecraft: Generally Capable Agents for Open-World Environments via Large Language Models with Text-based Knowledge and Memorycited by 289
  8. 2021Open-Ended Learning Leads to Generally Capable Agentscited by 182
  9. 2025Training Agents Inside of Scalable World Modelscited by 136
  10. 2023OMNI: Open-endedness via Models of human Notions of Interestingnesscited by 70
  11. 2023Auto MC-Reward: Automated Dense Reward Design with Large Language Models for Minecraftcited by 61
  12. 2023Skill Reinforcement Learning and Planning for Open-World Long-Horizon Taskscited by 36
23 more
  1. 2024S-Agents: Self-organizing Agents in Open-ended Environmentscited by 22
  2. 2022Avalon: A Benchmark for RL Generalization Using Procedurally Generated Worldscited by 17
  3. 2024Eurekaverse: Environment Curriculum Generation via Large Language Modelscited by 16
  4. 2026SPADE: Self-Play in Adaptive Synthetic Executable Environmentscited by 12
  5. 2023Cross-Episodic Curriculum for Transformer Agentscited by 10
  6. 2023Hieros: Hierarchical Imagination on Structured State Space Sequence World Modelscited by 8
  7. 2024Masked Generative Priors Improve World Models Sequence Modelling Capabilitiescited by 7
  8. 2025MAGELLAN: Metacognitive predictions of learning progress guide autotelic LLM agents in large goal spacescited by 7
  9. 2023Adaptive Neural Ranking Framework: Toward Maximized Business Goal for Cascade Ranking Systemscited by 6
  10. 2025Interleaved Multitask Learning with Energy Modulated Learning Progresscited by 3
  11. 2026Interestingness as an Inductive Heuristic for Future Compression Progresscited by 3
  12. 2022Teacher-student curriculum learning for reinforcement learningcited by 2
  13. 2023Learning Curricula in Open-Ended Worldscited by 2
  14. 2024Syllabus: Portable Curricula for Reinforcement Learning Agentscited by 2
  15. 2025DyMoDreamer: World Modeling with Dynamic Modulationcited by 2
  16. 2025Offline vs. Online Learning in Model-based RL: Lessons for Data Collection Strategiescited by 1
  17. 2026Scaling Curriculum Learning For Autonomous Drivingcited by 1
  18. 2024Massively Multiagent Minigames for Training Generalist Agentscited by 0
  19. 2025COCORELI: Enforcing Execution Preconditions for Reliable Collaborative Instruction Followingcited by 0
  20. 2026D'ej`aQ: Open-Ended Evolution of Diverse, Learnable and Verifiable Problemscited by 0
  21. 2026Efficient Unsupervised Environment Design through Hierarchical Policy Representation Learningcited by 0
  22. 2026Distributionally Robust Multi-Task Reinforcement Learning via Adaptive Task Samplingcited by 0
  23. 2026Anticipating the Consequences of Curriculum Decisions with Large Language Modelscited by 0