mlsanityData to 14 Oct 2026

Emergent Coordination Through Competition

arXiv 1902.07151 ↗First posted 19 Feb 2019cs.AIcited by 57

Not on the map. Why?

Abstract (arXiv)

We study the emergence of cooperative behaviors in reinforcement learning agents by introducing a challenging competitive multi-agent soccer environment with continuous simulated physics. We demonstrate that decentralized, population-based training with co-play can lead to a progression in agents' behaviors: from random, to simple ball chasing, and finally showing evidence of cooperation. Our study highlights several of the challenges encountered in large scale multi-agent training in continuous control. In particular, we demonstrate that the automatic optimization of simple shaping rewards, not themselves conducive to co-operative behavior, can lead to long-horizon team behavior. We further apply an evaluation scheme, grounded by game theoretic principals, that can assess agent performance in the absence of pre-defined evaluation tasks or human baselines.

Key facts. Cited by 57 papers in the map (data through 14 Oct 2026). First posted on arXiv 19 Feb 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 17; 2022: 12; 2023: 13; 2024: 8; 2025: 5; 2026: 2.

57 papers.

Sort:

By area:

  1. 2023Human-Timescale Adaptation in an Open-Ended Task Spacecited by 138
  2. 2023Learning Agile Soccer Skills for a Bipedal Robot with Deep Reinforcement Learningcited by 133
  3. 2021Scalable Evaluation of Multi-Agent Reinforcement Learning with Melting Potcited by 99
  4. 2021From Motor Control to Team Play in Simulated Humanoid Footballcited by 87
  5. 2022Automated Reinforcement Learning (AutoRL): A Survey and Open Problemscited by 70
  6. 2021Deep Multiagent Reinforcement Learning: Challenges and Directionscited by 42
  7. 2021Modelling Behavioural Diversity for Learning in Open-Ended Gamescited by 39
  8. 2023Evolutionary Reinforcement Learning: A Surveycited by 29
  9. 2023A Review of Cooperation in Multi-agent Learningcited by 29
  10. 2021On the Critical Role of Conventions in Adaptive Human-AI Collaborationcited by 27
  11. 2021Diverse Auto-Curriculum is Critical for Successful Real-World Multiagent Learning Systemscited by 24
  12. 2022Continuously Discovering Novel Strategies via Reward-Switching Policy Optimizationcited by 23
38 more
  1. 2021Pick Your Battles: Interaction Graphs as Population-Level Objectives for Strategic Diversitycited by 22
  2. 2021Deep Latent Competition: Learning to Race Using Visual Control Policies in Latent Spacecited by 20
  3. 2023TiZero: Mastering Multi-Agent Football with Curriculum Learning and Self-Playcited by 20
  4. 2021TiKick: Towards Playing Multi-agent Football Full Games from Single-agent Demonstrationscited by 18
  5. 2024Learning Robot Soccer from Egocentric Vision with Deep Reinforcement Learningcited by 17
  6. 2022NeuPL: Neural Population Learningcited by 14
  7. 2025Who's the Mole? Modeling and Detecting Intention-Hiding Malicious Agents in LLM-Based Multi-Agent Systemscited by 13
  8. 2023A Survey of Deep Learning in Sports Applications: Perception, Comprehension, and Decisioncited by 11
  9. 2021Tuning Mixed Input Hyperparameters on the Fly for Efficient Population Based AutoRLcited by 8
  10. 2023A Strategic Framework for Optimal Decisions in Football 1-vs-1 Shot-Taking Situations: An Integrated Approach of Machine Learning, Theory-Based Modeling, and Game Theorycited by 7
  11. 2024Reinforcement Learning Within the Classical Robotics Stack: A Case Study in Robot Soccercited by 7
  12. 2021How does AI play football? An analysis of RL and real-world football strategiescited by 6
  13. 2022MA-Dreamer: Coordination and communication through shared imaginationcited by 6
  14. 2023Learning Diverse Risk Preferences in Population-based Self-playcited by 6
  15. 2021Dealing with Non-Stationarity in MARL via Trust-Region Decompositioncited by 5
  16. 2021Two-stage training algorithm for AI robot soccercited by 4
  17. 2021rSoccer: A Framework for Studying Reinforcement Learning in Small and Very Small Size Robot Soccercited by 4
  18. 2022Lamarckian Platform: Pushing the Boundaries of Evolutionary Reinforcement Learning towards Asynchronous Commercial Gamescited by 4
  19. 2023Towards Skilled Population Curriculum for Multi-Agent Reinforcement Learningcited by 4
  20. 2024Neural Population Learning beyond Symmetric Zero-sum Gamescited by 4
  21. 2024Generalized Population-Based Training for Hyperparameter Optimization in Reinforcement Learningcited by 4
  22. 2025Agile Flight Emerges from Multi-Agent Competitive Racingcited by 4
  23. 2021Multi-Agent Coordination in Adversarial Environments through Signal Mediated Strategiescited by 3
  24. 2021Local Advantage Actor-Critic for Robust Multi-Agent Deep Reinforcement Learningcited by 3
  25. 2022Motivating Physical Activity via Competitive Human-Robot Interactioncited by 3
  26. 2022Beyond Rewards: a Hierarchical Perspective on Offline Multiagent Behavioral Analysiscited by 3
  27. 2022S2RL: Do We Really Need to Perceive All States in Deep Multi-Agent Reinforcement Learning?cited by 3
  28. 2023Iteratively Learn Diverse Strategies with State Distance Informationcited by 3
  29. 2024Enabling Multi-Robot Collaboration from Single-Human Guidancecited by 3
  30. 2022Developing, Evaluating and Scaling Learning Agents in Multi-Agent Environmentscited by 2
  31. 2024CompetEvo: Towards Morphological Evolution from Competitioncited by 2
  32. 2024Subequivariant Reinforcement Learning in 3D Multi-Entity Physical Environmentscited by 2
  33. 2025SoccerSynth-Detection: A Synthetic Dataset for Soccer Player Detectioncited by 2
  34. 2021Learning to Play Soccer From Scratch: Sample-Efficient Emergent Coordination through Curriculum-Learning and Competitioncited by 1
  35. 2022From Multi-agent to Multi-robot: A Scalable Training and Evaluation Platform for Multi-robot Reinforcement Learningcited by 1
  36. 2022Rethinking Trajectory Prediction via "Team Game"cited by 1
  37. 2023Diverse Policy Optimization for Structured Action Spacecited by 1
  38. 2025An Automated Reinforcement Learning Reward Design Framework with Large Language Model for Cooperative Platoon Coordinationcited by 1

50 most cited of 57 All 57 (JSON)