mlsanityData to 7 Oct 2026

Adaptable Agent Populations via a Generative Model of Policies

arXiv 2107.07506 ↗First posted 15 Jul 2021cs.LG cs.NEcited by 12

Not on the map. Why?

Abstract (arXiv)

In the natural world, life has found innumerable ways to survive and often thrive. Between and even within species, each individual is in some manner unique, and this diversity lends adaptability and robustness to life. In this work, we aim to learn a space of diverse and high-reward policies on any given environment. To this end, we introduce a generative model of policies, which maps a low-dimensional latent space to an agent policy space. Our method enables learning an entire population of agent policies, without requiring the use of separate policy parameters. Just as real world populations can adapt and evolve via natural selection, our method is able to adapt to changes in our environment solely by selecting for policies in latent space. We test our generative model's capabilities in a variety of environments, including an open-ended grid-world and a two-player soccer environment. Code, visualizations, and additional experiments can be found at https://kennyderek.github.io/adap/.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 16 of its 30 references that are in the map. First posted on arXiv 15 Jul 2021.

Builds on

16 of 30 references.

References · 16

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 734
  3. 2018QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 697
  4. 2015Illuminating search spaces by mapping elitescited by 406
  5. 2017Population Based Training of Neural Networkscited by 351
  6. 2019MAVEN: Multi-Agent Variational Explorationcited by 181
  7. 2019Goal-conditioned Imitation Learningcited by 159
  8. 2019Planning with Goal-Conditioned Policiescited by 144
  9. 2020ROMA: Multi-Agent Reinforcement Learning with Emergent Rolescited by 135
  10. 2017Stochastic Neural Networks for Hierarchical Reinforcement Learningcited by 124
  11. 2018Variational Option Discovery Algorithmscited by 121
  12. 2020Effective Diversity in Population Based Reinforcement Learningcited by 116
4 more
  1. 2020One Solution is Not All You Need: Few-Shot Extrapolation via Structured MaxEnt RLcited by 79
  2. 2018Diversity-Driven Exploration Strategy for Deep Reinforcement Learningcited by 65
  3. 2019Diversity-Inducing Policy Gradient: Using Maximum Mean Discrepancy to Find a Set of Diverse Policiescited by 34
  4. 2019Diverse Exploration via Conjugate Policies for Policy Gradient Methodscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 7; 2024: 0; 2025: 2; 2026: 1.

12 papers.

  1. 2023PECAN: Leveraging Policy Ensemble for Context-Aware Zero-Shot Human-AI Coordinationcited by 25
  2. 2023Adaptive Coordination in Social Embodied Rearrangementcited by 14
  3. 2023Diverse Conventions for Human-AI Collaborationcited by 14
  4. 2022DGPO: Discovering Multiple Strategies with Diversity-Guided Policy Optimizationcited by 10
  5. 2023Can Agents Run Relay Race with Strangers? Generalization of RL to Out-of-Distribution Trajectoriescited by 9
  6. 2023Communication-Robust Multi-Agent Learning by Adaptable Auxiliary Multi-Agent Adversary Generationcited by 6
  7. 2025Improving Human-AI Coordination through Online Adversarial Training and Generative Modelscited by 3
  8. 2023Heterogeneous Social Value Orientation Leads to Meaningful Diversity in Sequential Social Dilemmascited by 2
  9. 2026Shaping Zero-Shot Coordination via State Blockingcited by 2
  10. 2022Open-Ended Diverse Solution Discovery with Regulated Behavior Patterns for Cross-Domain Adaptationcited by 1
  11. 2023Diverse Policy Optimization for Structured Action Spacecited by 1
  12. 2025Algorithmic Prompt Generation for Diverse Human-like Teaming and Communication with Large Language Modelscited by 1