mlsanityData to 7 Oct 2026

Human-level performance in first-person multiplayer games with population-based deep reinforcement learning

arXiv 1807.01281 ↗First posted 3 Jul 2018cs.LG cs.AI stat.MLcited by 260

Not on the map. Why?

Abstract (arXiv)

Recent progress in artificial intelligence through reinforcement learning (RL) has shown great success on increasingly complex single-agent environments and two-player turn-based games. However, the real-world contains multiple agents, each learning and acting independently to cooperate and compete with other agents, and environments reflecting this degree of complexity remain an open challenge. In this work, we demonstrate for the first time that an agent can achieve human-level in a popular 3D multiplayer first-person video game, Quake III Arena Capture the Flag, using only pixels and game points as input. These results were achieved by a novel two-tier optimisation process in which a population of independent RL agents are trained concurrently from thousands of parallel matches with agents playing in teams together and against each other on randomly generated environments. Each agent in the population learns its own internal reward signal to complement the sparse delayed reward from winning, and selects actions using a novel temporally hierarchical representation that enables the agent to reason at multiple timescales. During game-play, these agents display human-like behaviours such as navigating, following, and defending based on a rich learned representation that is shown to encode high-level game knowledge. In an extensive tournament-style evaluation the trained agents exceeded the win-rate of strong human players both as teammates and opponents, and proved far stronger than existing state-of-the-art agents. These results demonstrate a significant jump in the capabilities of artificial agents, bringing us closer to the goal of human-level intelligence.

Key facts. Cited by 260 papers in the map (data through 7 Oct 2026). First posted on arXiv 3 Jul 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2018: 1; 2019: 0; 2020: 2; 2021: 66; 2022: 57; 2023: 59; 2024: 36; 2025: 22; 2026: 17.

260 papers.

Sort:

By area:

  1. 2018Scalable agent alignment via reward modeling: a research directioncited by 438
  2. 2021Brax -- A Differentiable Physics Engine for Large Scale Rigid Body Simulationcited by 375
  3. 2022Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videoscited by 338
  4. 2023Rewarded soups: towards Pareto-optimal alignment by interpolating weights fine-tuned on diverse rewardscited by 232
  5. 2023Building Cooperative Embodied Agents Modularly with Large Language Modelscited by 231
  6. 2023Habitat 3.0: A Co-Habitat for Humans, Avatars and Robotscited by 220
  7. 2021Open-Ended Learning Leads to Generally Capable Agentscited by 177
  8. 2022From Attribution Maps to Human-Understandable Explanations through Concept Relevance Propagationcited by 141
  9. 2021Collaborating with Humans without Human Datacited by 139
  10. 2023Human-Timescale Adaptation in an Open-Ended Task Spacecited by 136
  11. 2022Mastering the Game of Stratego with Model-Free Multiagent Reinforcement Learningcited by 129
  12. 2025REASONING GYM: Reasoning Environments for Reinforcement Learning with Verifiable Rewardscited by 121
38 more
  1. 2023Generative agent-based modeling with actions grounded in physical, social, or digital space using Concordiacited by 119
  2. 2023Avalon's Game of Thoughts: Battle Against Deception through Recursive Contemplationcited by 102
  3. 2021Scalable Evaluation of Multi-Agent Reinforcement Learning with Melting Potcited by 97
  4. 2021From Motor Control to Team Play in Simulated Humanoid Footballcited by 87
  5. 2022Transformers as Meta-Learners for Implicit Neural Representationscited by 79
  6. 2021Maximum Entropy Population-Based Training for Zero-Shot Human-AI Coordinationcited by 69
  7. 2022Automated Reinforcement Learning (AutoRL): A Survey and Open Problemscited by 69
  8. 2023Auto MC-Reward: Automated Dense Reward Design with Large Language Models for Minecraftcited by 60
  9. 2021Counter-Strike Deathmatch with Large-Scale Behavioural Cloningcited by 49
  10. 2024COMBO: Compositional World Models for Embodied Multi-Agent Cooperationcited by 47
  11. 2023A Survey on Causal Reinforcement Learningcited by 43
  12. 2021Deep Multiagent Reinforcement Learning: Challenges and Directionscited by 42
  13. 2020Energy-Based Imitation Learningcited by 39
  14. 2021Modelling Behavioural Diversity for Learning in Open-Ended Gamescited by 39
  15. 2022Melting Pot 2.0cited by 37
  16. 2021Alchemy: A benchmark and analysis toolkit for meta-reinforcement learning agentscited by 35
  17. 2021Discovering Diverse Multi-Agent Strategic Behavior via Reward Randomizationcited by 35
  18. 2021XDO: A Double Oracle Algorithm for Extensive-Form Gamescited by 34
  19. 2022K-level Reasoning for Zero-Shot Coordination in Hanabicited by 30
  20. 2023MAESTRO: Open-Ended Environment Design for Multi-Agent Reinforcement Learningcited by 30
  21. 2023Multi-agent Reinforcement Learning: A Comprehensive Surveycited by 30
  22. 2024Multi-Agent Reinforcement Learning for Autonomous Driving: A Surveycited by 30
  23. 2023Decision-Oriented Dialogue for Human-AI Collaborationcited by 29
  24. 2023A Review of Cooperation in Multi-agent Learningcited by 27
  25. 2024A Survey on Self-play Methods in Reinforcement Learningcited by 26
  26. 2021Evaluating the Robustness of Collaborative Agentscited by 25
  27. 2021Diverse Auto-Curriculum is Critical for Successful Real-World Multiagent Learning Systemscited by 24
  28. 2023Evolutionary Reinforcement Learning: A Surveycited by 24
  29. 2023Neurosymbolic Reinforcement Learning and Planning: A Surveycited by 24
  30. 2021MALib: A Parallel Framework for Population-based Multi-agent Reinforcement Learningcited by 23
  31. 2021Deep Reinforcement Learning Versus Evolution Strategies: A Comparative Surveycited by 23
  32. 2022Continuously Discovering Novel Strategies via Reward-Switching Policy Optimizationcited by 23
  33. 2022On the Limitations of Elo: Real-World Games, are Transitive, not Additivecited by 23
  34. 2022Law Informs Code: A Legal Informatics Approach to Aligning Artificial Intelligence with Humanscited by 23
  35. 2023NOPA: Neurally-guided Online Probabilistic Assistance for Building Socially Intelligent Home Assistantscited by 23
  36. 2024Can VLMs Play Action Role-Playing Games? Take Black Myth Wukong as a Study Casecited by 23
  37. 2021Pick Your Battles: Interaction Graphs as Population-Level Objectives for Strategic Diversitycited by 22
  38. 2021Quantifying the effects of environment and population diversity in multi-agent reinforcement learningcited by 21

50 most cited of 260 All 260 (JSON)