mlsanityData to 7 Oct 2026

Paired Open-Ended Trailblazer (POET): Endlessly Generating Increasingly Complex and Diverse Learning Environments and Their Solutions

arXiv 1901.01753 ↗First posted 7 Jan 2019cs.NEcited by 182

Not on the map. Why?

Cited by 23 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

While the history of machine learning so far largely encompasses a series of problems posed by researchers and algorithms that learn their solutions, an important question is whether the problems themselves can be generated by the algorithm at the same time as they are being solved. Such a process would in effect build its own diverse and expanding curricula, and the solutions to problems at various stages would become stepping stones towards solving even more challenging problems later in the process. The Paired Open-Ended Trailblazer (POET) algorithm introduced in this paper does just that: it pairs the generation of environmental challenges and the optimization of agents to solve those challenges. It simultaneously explores many different paths through the space of possible problems and solutions and, critically, allows these stepping-stone solutions to transfer between problems if better, catalyzing innovation. The term open-ended signifies the intriguing potential for algorithms like POET to continue to create novel and increasingly complex capabilities without bound. Our results show that POET produces a diverse range of sophisticated behaviors that solve a wide range of environmental challenges, many of which cannot be solved by direct optimization alone, or even through a direct-path curriculum-building control algorithm introduced to highlight the critical role of open-endedness in solving ambitious challenges. The ability to transfer solutions from one environment to another proves essential to unlocking the full potential of the system as a whole, demonstrating the unpredictable nature of fortuitous stepping stones. We hope that POET will inspire a new push towards open-ended discovery across many domains, where algorithms like POET can blaze a trail through their interesting possible manifestations and solutions.

Key facts. Cited by 182 papers in the map (data through 7 Oct 2026). First posted on arXiv 7 Jan 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 26; 2022: 22; 2023: 28; 2024: 21; 2025: 41; 2026: 44.

182 papers.

Sort:

By area:

  1. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
  2. 2022MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledgecited by 469
  3. 2021Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learningcited by 395
  4. 2025Absolute Zero: Reinforced Self-play Reasoning with Zero Datacited by 323
  5. 2025Darwin Godel Machine: Open-Ended Evolution of Self-Improving Agentscited by 294
  6. 2022Evolving Curricula with Regret-Based Environment Designcited by 166
  7. 2021A Survey of Zero-shot Generalisation in Deep Reinforcement Learningcited by 164
  8. 2023Human-Timescale Adaptation in an Open-Ended Task Spacecited by 136
  9. 2023GenSim: Generating Robotic Simulation Tasks via Large Language Modelscited by 120
  10. 2021Replay-Guided Adversarial Environment Designcited by 108
  11. 2025Self-Evolving Curriculum for LLM Reasoningcited by 107
  12. 2021Scalable Evaluation of Multi-Agent Reinforcement Learning with Melting Potcited by 97
38 more
  1. 2025Efficient Reinforcement Finetuning via Adaptive Curriculum Learningcited by 88
  2. 2024Craftax: A Lightning-Fast Benchmark for Open-Ended Reinforcement Learningcited by 80
  3. 2024Open-Endedness is Essential for Artificial Superhuman Intelligencecited by 79
  4. 2021Asymmetric self-play for automatic goal discovery in robotic manipulationcited by 71
  5. 2023OMNI: Open-endedness via Models of human Notions of Interestingnesscited by 66
  6. 2025Improving Data Efficiency for LLM Reinforcement Fine-tuning Through Difficulty-targeted Online Data Selection and Rollout Replaycited by 52
  7. 2025DUMP: Automated Distribution-Level Curriculum Learning for RL-based LLM Post-trainingcited by 45
  8. 2025RLVE: Scaling Up Reinforcement Learning for Language Models with Adaptive Verifiable Environmentscited by 43
  9. 2022Deep Surrogate Assisted Generation of Environmentscited by 40
  10. 2024Kinetix: Investigating the Training of General Agents through Open-Ended Physics-Based Control Taskscited by 40
  11. 2023Open-World Multi-Task Control Through Goal-Aware Representation Learning and Adaptive Horizon Predictioncited by 39
  12. 2021Adversarial Reinforcement Learning for Procedural Content Generationcited by 36
  13. 2021Multi-task curriculum learning in a complex, visual, hard-exploration domain: Minecraftcited by 34
  14. 2022Environment Generation for Zero-Shot Compositional Reinforcement Learningcited by 32
  15. 2023MAESTRO: Open-Ended Environment Design for Multi-Agent Reinforcement Learningcited by 30
  16. 2024No Regrets: Investigating and Improving Regret Approximations for Curriculum Discoverycited by 29
  17. 2025Training a Generally Curious Agentcited by 29
  18. 2025SIMA 2: A Generalist Embodied Agent for Virtual Worldscited by 26
  19. 2025Large Language Models Miss the Multi-Agent Markcited by 25
  20. 2021Quantifying the effects of environment and population diversity in multi-agent reinforcement learningcited by 21
  21. 2022Blocks Assemble! Learning to Assemble with Large-Scale Structured Reinforcement Learningcited by 21
  22. 2023Latent Exploration for Reinforcement Learningcited by 21
  23. 2022Bayesian Generational Population-Based Trainingcited by 20
  24. 2024A social path to human-like artificial intelligencecited by 20
  25. 2022CLUTR: Curriculum Learning via Unsupervised Task Representation Learningcited by 19
  26. 2025Questioning Representational Optimism in Deep Learning: The Fractured Entangled Representation Hypothesiscited by 19
  27. 2021TeachMyAgent: a Benchmark for Automatic Curriculum Learning in Deep RLcited by 18
  28. 2022Emergent Bartering Behaviour in Multi-Agent Reinforcement Learningcited by 18
  29. 2024Word2World: Generating Stories and Worlds through Large Language Modelscited by 18
  30. 2021Distributionally Robust Policy Learning via Adversarial Environment Generationcited by 17
  31. 2023Stabilizing Unsupervised Environment Design with a Learned Adversarycited by 17
  32. 2022Grounding Aleatoric Uncertainty for Unsupervised Environment Designcited by 16
  33. 2023Discovering General Reinforcement Learning Algorithms with Adversarial Environment Designcited by 16
  34. 2024Refining Minimax Regret for Unsupervised Environment Designcited by 16
  35. 2022The Effectiveness of World Models for Continual Reinforcement Learningcited by 15
  36. 2022It Takes Four to Tango: Multiagent Selfplay for Automatic Curriculum Generationcited by 14
  37. 2024Eurekaverse: Environment Curriculum Generation via Large Language Modelscited by 14
  38. 2021Adversarial Environment Generation for Learning to Navigate the Webcited by 13

50 most cited of 182 All 182 (JSON)