Data to 14 Oct 2026

Emergent Complexity via Multi-Agent Competition

arXiv 1710.03748 ↗First posted 10 Oct 2017cs.AIcited by 120

Not on the map. Why?

Abstract (arXiv)

Reinforcement learning algorithms can train agents that solve problems in complex, interesting environments. Normally, the complexity of the trained agent is closely related to the complexity of the environment. This suggests that a highly capable agent requires a complex environment for training. In this paper, we point out that a competitive multi-agent environment trained with self-play can produce behaviors that are far more complex than the environment itself. We also point out that such environments come with a natural curriculum, because for any skill level, an environment full of agents of this level will have the right level of difficulty. This work introduces several competitive multi-agent environments where agents compete in a 3D world with simulated physics. The trained agents learn a wide variety of complex and interesting skills, even though the environment themselves are relatively simple. The skills include behaviors such as running, blocking, ducking, tackling, fooling opponents, kicking, and defending using both arms and legs. A highlight of the learned behaviors can be found here: https://goo.gl/eR7fbX

Key facts. Cited by 120 papers in the map (data through 14 Oct 2026). First posted on arXiv 10 Oct 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 17; 2022: 23; 2023: 20; 2024: 23; 2025: 17; 2026: 20.

120 papers.

Sort:

By area:

2024Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Modelscited by 557
2023Learning Agile Soccer Skills for a Bipedal Robot with Deep Reinforcement Learningcited by 133
2021Scalable Evaluation of Multi-Agent Reinforcement Learning with Melting Potcited by 99
2021From Motor Control to Team Play in Simulated Humanoid Footballcited by 87
2021Asymmetric self-play for automatic goal discovery in robotic manipulationcited by 72
2021BACKDOORL: Backdoor Attack against Competitive Reinforcement Learningcited by 68
2021Evaluation of Human-AI Teams for Learned and Rule-Based Agents in Hanabicited by 40
2023PMP: Learning to Physically Interact with Environments using Part-wise Motion Priorscited by 38
2021XDO: A Double Oracle Algorithm for Extensive-Form Gamescited by 34
2023MAESTRO: Open-Ended Environment Design for Multi-Agent Reinforcement Learningcited by 31
2024A Survey on Self-play Methods in Reinforcement Learningcited by 26
2022BAFFLE: Hiding Backdoors in Offline Reinforcement Learning Datasetscited by 25

38 more

2022On the Limitations of Elo: Real-World Games, are Transitive, not Additivecited by 23
2025The Coming Crisis of Multi-Agent Misalignment: AI Alignment Must Be a Dynamic and Social Processcited by 21
2025Virtual Agent Economiescited by 21
2021Deep Latent Competition: Learning to Race Using Visual Control Policies in Latent Spacecited by 20
2024A social path to human-like artificial intelligencecited by 20
2022PolicyCleanse: Backdoor Detection and Mitigation in Reinforcement Learningcited by 19
2021Collaborative Visual Navigationcited by 18
2022ALMA: Hierarchical Learning for Composite Multi-Agent Taskscited by 18
2024Zero-Shot Reinforcement Learning via Function Encoderscited by 18
2021ROMAX: Certifiably Robust Deep Multiagent Reinforcement Learning via Convex Relaxationcited by 17
2024Learning Robot Soccer from Egocentric Vision with Deep Reinforcement Learningcited by 17
2024Self-Play with Adversarial Critic: Provable and Scalable Offline Alignment for Language Modelscited by 16
2024Eurekaverse: Environment Curriculum Generation via Large Language Modelscited by 16
2022It Takes Four to Tango: Multiagent Selfplay for Automatic Curriculum Generationcited by 14
2024Toward Optimal LLM Alignments Using Two-Player Gamescited by 13
2024A Review of Nine Physics Engines for Reinforcement Learning Researchcited by 12
2025Distributional AGI Safetycited by 12
2022Reinforcement Learning and Bandits for Speech and Language Processing: Tutorial, Review and Outlookcited by 10
2023Learning to Identify Critical States for Reinforcement Learning from Videoscited by 9
2024Discovering Sensorimotor Agency in Cellular Automata using Diversity Searchcited by 9
2026Triplets Better Than Pairs: Towards Stable and Effective Self-Play Fine-Tuning for LLMscited by 9
2021Local Patch AutoAugment with Multi-Agent Collaborationcited by 8
2022GraspARL: Dynamic Grasping via Adversarial Reinforcement Learningcited by 8
2022Curriculum-based Asymmetric Multi-task Reinforcement Learningcited by 8
2024From Text to Life: On the Reciprocal Relationship between Artificial Life and Large Language Modelscited by 8
2024SRAP-Agent: Simulating and Optimizing Scarce Resource Allocation Policy with LLM-based Agentcited by 8
2022DouZero+: Improving DouDizhu AI by Opponent Modeling and Coach-guided Learningcited by 7
2023Rethinking Adversarial Policies: A Generalized Attack Formulation and Provable Defense in RLcited by 7
2024SUB-PLAY: Adversarial Policies against Partially Observed Multi-Agent Reinforcement Learning Systemscited by 7
2026Digital Red Queen: Adversarial Program Evolution in Core War with LLMscited by 7
2026FutureSim: Replaying World Events to Evaluate Adaptive Agentscited by 7
2023Learning Diverse Risk Preferences in Population-based Self-playcited by 6
2023Fictitious Cross-Play: Learning Global Nash Equilibrium in Mixed Cooperative-Competitive Gamescited by 6
2021Reinforcement Learning In Two Player Zero Sum Simultaneous Action Gamescited by 4
2022Reducing Exploitability with Population Based Trainingcited by 4
2022DIAMBRA Arena: a New Reinforcement Learning Platform for Research and Experimentationcited by 4
2022Revealing Robust Oil and Gas Company Macro-Strategies using Deep Multi-Agent Reinforcement Learningcited by 4
2025RIVAL: Reinforcement Learning with Iterative and Adversarial Optimization for Machine Translationcited by 4

50 most cited of 120 All 120 (JSON)