mlsanityData to 14 Oct 2026

Diversity Induced Environment Design via Self-Play

arXiv 2302.02119 ↗First posted 4 Feb 2023cs.AI cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Recent work on designing an appropriate distribution of environments has shown promise for training effective generally capable agents. Its success is partly because of a form of adaptive curriculum learning that generates environment instances (or levels) at the frontier of the agent's capabilities. However, such an environment design framework often struggles to find effective levels in challenging design spaces and requires costly interactions with the environment. In this paper, we aim to introduce diversity in the Unsupervised Environment Design (UED) framework. Specifically, we propose a task-agnostic method to identify observed/hidden states that are representative of a given level. The outcome of this method is then utilized to characterize the diversity between two levels, which as we show can be crucial to effective performance. In addition, to improve sampling efficiency, we incorporate the self-play technique that allows the environment generator to automatically generate environments that are of great benefit to the training agent. Quantitatively, our approach, Diversity-induced Environment Design via Self-Play (DivSP), shows compelling performance over existing methods.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 24 of its 36 references that are in the map. First posted on arXiv 4 Feb 2023.

Builds on

24 of 36 references.

References · 24

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 2,026
  3. 2016OpenAI Gymcited by 1,736
  4. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,498
  5. 2015End-to-End Training of Deep Visuomotor Policiescited by 875
  6. 2019Solving Rubik's Cube with a Robot Handcited by 734
  7. 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 585
  8. 2020Emergent Complexity and Zero-shot Transfer via Unsupervised Environment Designcited by 275
  9. 2017Reverse Curriculum Generation for Reinforcement Learningcited by 198
  10. 2020Prioritized Level Replaycited by 198
  11. 2019Paired Open-Ended Trailblazer (POET): Endlessly Generating Increasingly Complex and Diverse Learning Environments and Their Solutionscited by 192
  12. 2022Evolving Curricula with Regret-Based Environment Designcited by 173
12 more
  1. 2020Automatic Curriculum Learning For Deep RL: A Short Surveycited by 150
  2. 2017Intrinsic Motivation and Automatic Curricula via Asymmetric Self-Playcited by 141
  3. 2019Teacher algorithms for curriculum learning of Deep RL in continuously parameterized environmentscited by 119
  4. 2020Effective Diversity in Population Based Reinforcement Learningcited by 119
  5. 2021Replay-Guided Adversarial Environment Designcited by 111
  6. 2020When Do Curricula Work?cited by 106
  7. 2020Automatic Curriculum Learning through Value Disagreementcited by 87
  8. 2020Improving Generalization in Reinforcement Learning with Mixture Regularizationcited by 81
  9. 2020Automatic Data Augmentation for Generalization in Deep Reinforcement Learningcited by 71
  10. 2018Theory of Curriculum Learning, with Convex Loss Functionscited by 33
  11. 2022It Takes Four to Tango: Multiagent Selfplay for Automatic Curriculum Generationcited by 14
  12. 2021CLAIM: Curriculum Learning Policy for Influence Maximization in Unknown Social Networkscited by 7

Built on by

Not cited yet.