mlsanityData to 7 Oct 2026

Emergent Tool Use From Multi-Agent Autocurricula

arXiv 1909.07528 ↗First posted 17 Sep 2019cs.LG cs.AI cs.MAcited by 308

Not on the map. Why?

Cited by 25 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Through multi-agent competition, the simple objective of hide-and-seek, and standard reinforcement learning algorithms at scale, we find that agents create a self-supervised autocurriculum inducing multiple distinct rounds of emergent strategy, many of which require sophisticated tool use and coordination. We find clear evidence of six emergent phases in agent strategy in our environment, each of which creates a new pressure for the opposing team to adapt; for instance, agents learn to build multi-object shelters using moveable boxes which in turn leads to agents discovering that they can overcome obstacles using ramps. We further provide evidence that multi-agent competition may scale better with increasing environment complexity and leads to behavior that centers around far more human-relevant skills than other self-supervised reinforcement learning methods such as intrinsic motivation. Finally, we propose transfer and fine-tuning as a way to quantitatively evaluate targeted capabilities, and we compare hide-and-seek agents to both intrinsic motivation and random initialization baselines in a suite of domain-specific intelligence tests.

Key facts. Cited by 308 papers in the map (data through 7 Oct 2026). First posted on arXiv 17 Sep 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 65; 2022: 67; 2023: 59; 2024: 40; 2025: 49; 2026: 28.

308 papers.

Sort:

By area:

  1. 2021The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Gamescited by 903
  2. 2022Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videoscited by 338
  3. 2025Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscationcited by 338
  4. 2022The Effects of Reward Misspecification: Mapping and Mitigating Misaligned Modelscited by 312
  5. 2023Tool Learning with Foundation Modelscited by 302
  6. 2023Building Cooperative Embodied Agents Modularly with Large Language Modelscited by 231
  7. 2023Habitat 3.0: A Co-Habitat for Humans, Avatars and Robotscited by 220
  8. 2023An Overview of Catastrophic AI Riskscited by 214
  9. 2021Open-Ended Learning Leads to Generally Capable Agentscited by 177
  10. 2021Policy Finetuning: Bridging Sample-Efficient Offline and Online Reinforcement Learningcited by 157
  11. 2023Human-Timescale Adaptation in an Open-Ended Task Spacecited by 136
  12. 2021V-Learning -- A Simple, Efficient, Decentralized Algorithm for Multiagent RLcited by 111
38 more
  1. 2023Learning Agile Soccer Skills for a Bipedal Robot with Deep Reinforcement Learningcited by 111
  2. 2022Is Power-Seeking AI an Existential Risk?cited by 101
  3. 2021Scalable Evaluation of Multi-Agent Reinforcement Learning with Melting Potcited by 97
  4. 2021From Motor Control to Team Play in Simulated Humanoid Footballcited by 87
  5. 2024Open-Endedness is Essential for Artificial Superhuman Intelligencecited by 79
  6. 2025Open Challenges in Multi-Agent Security: Towards Secure Systems of Interacting AI Agentscited by 66
  7. 2021When Can We Learn General-Sum Markov Games with a Large Number of Players Sample-Efficiently?cited by 63
  8. 2022VMAS: A Vectorized Multi-Agent Simulator for Collective Robot Learningcited by 63
  9. 2021Robot Learning from Randomized Simulations: A Reviewcited by 56
  10. 2021Contrasting Centralized and Decentralized Critics in Multi-Agent Reinforcement Learningcited by 54
  11. 2024Learning diverse attacks on large language models for robust red-teaming and safety tuningcited by 54
  12. 2021Attention, please! A survey of Neural Attention Models in Deep Learningcited by 50
  13. 2023A Survey on Transformers in Reinforcement Learningcited by 50
  14. 2021The Power of Exploiter: Provable Multi-Agent RL in Large State Spacescited by 48
  15. 2024COMBO: Compositional World Models for Embodied Multi-Agent Cooperationcited by 47
  16. 2022Retrospectives on the Embodied AI Workshopcited by 46
  17. 2025RLVE: Scaling Up Reinforcement Learning for Language Models with Adaptive Verifiable Environmentscited by 43
  18. 2021Deep Multiagent Reinforcement Learning: Challenges and Directionscited by 42
  19. 2024Optima: Optimizing Effectiveness and Efficiency for LLM-Based Multi-Agent Systemcited by 40
  20. 2021Modelling Behavioural Diversity for Learning in Open-Ended Gamescited by 39
  21. 2021Augmenting Automated Game Testing with Deep Reinforcement Learningcited by 38
  22. 2022Melting Pot 2.0cited by 37
  23. 2021Discovering Diverse Multi-Agent Strategic Behavior via Reward Randomizationcited by 35
  24. 2021Learning to Simulate Self-Driven Particles System with Coordinated Policy Optimizationcited by 33
  25. 2022Revisiting Some Common Practices in Cooperative Multi-Agent Reinforcement Learningcited by 32
  26. 2022LEAGUE: Guided Skill Learning and Abstraction for Long-Horizon Manipulationcited by 32
  27. 2022Disentangling Transfer in Continual Reinforcement Learningcited by 31
  28. 2023Emergence of Maps in the Memories of Blind Navigation Agentscited by 31
  29. 2021Towards Distraction-Robust Active Visual Trackingcited by 30
  30. 2023MAESTRO: Open-Ended Environment Design for Multi-Agent Reinforcement Learningcited by 30
  31. 2022Reward Gaming in Conditional Text Generationcited by 29
  32. 2023Breaking the Curse of Multiagency: Provably Efficient Decentralized Multi-Agent RL with Function Approximationcited by 29
  33. 2021On the Critical Role of Conventions in Adaptive Human-AI Collaborationcited by 26
  34. 2021TempoRL: Learning When to Actcited by 25
  35. 2021Variational Automatic Curriculum Learning for Sparse-Reward Cooperative Multi-Agent Problemscited by 25
  36. 2021Diverse Auto-Curriculum is Critical for Successful Real-World Multiagent Learning Systemscited by 24
  37. 2023The SocialAI School: Insights from Developmental Psychology Towards Artificial Socio-Cultural Agentscited by 24
  38. 2022Continuously Discovering Novel Strategies via Reward-Switching Policy Optimizationcited by 23

50 most cited of 308 All 308 (JSON)