mlsanityData to 7 Oct 2026

TiZero: Mastering Multi-Agent Football with Curriculum Learning and Self-Play

arXiv 2302.07515 ↗First posted 15 Feb 2023cs.AI cs.LG cs.MAcited by 18

Not on the map. Why?

Abstract (arXiv)

Multi-agent football poses an unsolved challenge in AI research. Existing work has focused on tackling simplified scenarios of the game, or else leveraging expert demonstrations. In this paper, we develop a multi-agent system to play the full 11 vs. 11 game mode, without demonstrations. This game mode contains aspects that present major challenges to modern reinforcement learning algorithms; multi-agent coordination, long-term planning, and non-transitivity. To address these challenges, we present TiZero; a self-evolving, multi-agent system that learns from scratch. TiZero introduces several innovations, including adaptive curriculum learning, a novel self-play strategy, and an objective that optimizes the policies of multiple agents jointly. Experimentally, it outperforms previous systems by a large margin on the Google Research Football environment, increasing win rates by over 30%. To demonstrate the generality of TiZero's innovations, they are assessed on several environments beyond football; Overcooked, Multi-agent Particle-Environment, Tic-Tac-Toe and Connect-Four.

Key facts. Cited by 18 papers in the map (data through 7 Oct 2026). Builds on 42 of its 78 references that are in the map. First posted on arXiv 15 Feb 2023.

Builds on

42 of 78 references.

References · 42

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2017Proximal Policy Optimization Algorithmscited by 14,441
  4. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 1,916
  5. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,519
  6. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,084
  7. 2013Exact solutions to the nonlinear dynamics of learning in deep linear neural networkscited by 748
  8. 2017Counterfactual Multi-Agent Policy Gradientscited by 733
  9. 2017Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithmcited by 723
  10. 2018QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 697
  11. 2017Value-Decomposition Networks For Cooperative Multi-Agent Learningcited by 490
  12. 2019Leveraging Procedural Generation to Benchmark Reinforcement Learningcited by 478
30 more
  1. 2019QTRAN: Learning to Factorize with Transformation for Cooperative Multi-Agent Reinforcement Learningcited by 372
  2. 2020Agent57: Outperforming the Atari Human Benchmarkcited by 300
  3. 2020QPLEX: Duplex Dueling Multi-Agent Q-Learningcited by 267
  4. 2017A Unified Game-Theoretic Approach to Multiagent Reinforcement Learningcited by 265
  5. 2019Google Research Football: A Novel Reinforcement Learning Environmentcited by 257
  6. 2020First return, then explorecited by 256
  7. 2013The Complexity of Decentralized Control of Markov Decision Processescited by 225
  8. 2016ViZDoom: A Doom-based AI Research Platform for Visual Reinforcement Learningcited by 178
  9. 2022Multi-Agent Reinforcement Learning is a Sequence Modeling Problemcited by 153
  10. 2020RODE: Learning Roles to Decompose Multi-Agent Taskscited by 125
  11. 2020Effective Diversity in Population Based Reinforcement Learningcited by 116
  12. 2019Open-ended Learning in Symmetric Zero-sum Gamescited by 96
  13. 2021Celebrating Diversity in Shared Multi-Agent Reinforcement Learningcited by 96
  14. 2021From Motor Control to Team Play in Simulated Humanoid Footballcited by 87
  15. 2020Towards Playing Full MOBA Games with Deep Reinforcement Learningcited by 78
  16. 2020Real World Games Look Like Spinning Topscited by 66
  17. 2021DouZero: Mastering DouDizhu with Self-Play Deep Reinforcement Learningcited by 65
  18. 2019Emergent Coordination Through Competitioncited by 57
  19. 2021Counter-Strike Deathmatch with Large-Scale Behavioural Cloningcited by 49
  20. 2021On Bonus-Based Exploration Methods in the Arcade Learning Environmentcited by 37
  21. 2020BeBold: Exploration Beyond the Boundary of Explored Regionscited by 32
  22. 2022Revisiting Some Common Practices in Cooperative Multi-Agent Reinforcement Learningcited by 32
  23. 2020Too many cooks: Bayesian inference for coordinating multi-agent collaborationcited by 29
  24. 2021TiKick: Towards Playing Multi-agent Football Full Games from Single-agent Demonstrationscited by 18
  25. 2022PerfectDou: Dominating DouDizhu with Perfect Information Distillationcited by 15
  26. 2021Unifying Behavioral and Response Diversity for Open-ended Learning in Zero-sum Gamescited by 14
  27. 2022NeuPL: Neural Population Learningcited by 14
  28. 2022DGPO: Discovering Multiple Strategies with Diversity-Guided Policy Optimizationcited by 10
  29. 2021Learning Diverse Policies in MOBA Games via Macro-Goalscited by 7
  30. 2021rSoccer: A Framework for Studying Reinforcement Learning in Small and Very Small Size Robot Soccercited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 6; 2025: 7; 2026: 2.

18 papers.

Sort:

By area:

  1. 2024A Survey on Self-play Methods in Reinforcement Learningcited by 26
  2. 2024N-Agent Ad Hoc Teamworkcited by 15
  3. 2023A Survey of Deep Learning in Sports Applications: Perception, Comprehension, and Decisioncited by 11
  4. 2025ROTATE: Regret-driven Open-ended Training for Ad Hoc Teamworkcited by 6
  5. 2024Reinforcement Learning Within the Classical Robotics Stack: A Case Study in Robot Soccercited by 5
  6. 2024Multi-Agent Diagnostics for Robustness via Illuminated Diversitycited by 4
  7. 2024The impact of behavioral diversity in multi-agent reinforcement learningcited by 3
  8. 2025SPECTra: Scalable Multi-Agent Reinforcement Learning with Permutation-Free Networkscited by 2
  9. 2023OpenRL: A Unified Reinforcement Learning Frameworkcited by 1
  10. 2025Improving Sample Efficiency in Multi-Agent Reinforcement Learning for Simulated Football Games via Explorationcited by 1
  11. 2023DCIR: Dynamic Consistency Intrinsic Reward for Multi-Agent Reinforcement Learningcited by 0
  12. 2024Transformer Guided Coevolution: Improved Team Selection in Multiagent Adversarial Team Gamescited by 0
6 more
  1. 2025Generalizable Agent Modeling for Agent Collaboration-Competition Adaptation with Multi-Retrieval and Dynamic Generationcited by 0
  2. 2025Understanding Action Effects through Instrumental Empowerment in Multi-Agent Reinforcement Learningcited by 0
  3. 2025Expandable Decision-Making States for Multi-Agent Deep Reinforcement Learning in Soccer Tactical Analysiscited by 0
  4. 2025ReCollab: Retrieval-Augmented LLMs for Cooperative Ad-hoc Teammate Modelingcited by 0
  5. 2026NePPO: Near-Potential Policy Optimization for General-Sum Multi-Agent Reinforcement Learningcited by 0
  6. 2026Reinforcement Learning Enabled Adaptive Multi-Task Control for Bipedal Soccer Robotscited by 0