mlsanityData to 7 Oct 2026

Google Research Football: A Novel Reinforcement Learning Environment

arXiv 1907.11180 ↗First posted 25 Jul 2019cs.LG stat.MLcited by 257

Not on the map. Why?

Abstract (arXiv)

Recent progress in the field of reinforcement learning has been accelerated by virtual learning environments such as video games, where novel algorithms and ideas can be quickly tested in a safe and reproducible manner. We introduce the Google Research Football Environment, a new reinforcement learning environment where agents are trained to play football in an advanced, physics-based 3D simulator. The resulting environment is challenging, easy to use and customize, and it is available under a permissive open-source license. In addition, it provides support for multiplayer and multi-agent experiments. We propose three full-game scenarios of varying difficulty with the Football Benchmarks and report baseline results for three commonly used reinforcement algorithms (IMPALA, PPO, and Ape-X DQN). We also provide a diverse set of simpler scenarios with the Football Academy and showcase several promising research directions.

Key facts. Cited by 257 papers in the map (data through 7 Oct 2026). First posted on arXiv 25 Jul 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 30; 2022: 42; 2023: 59; 2024: 48; 2025: 49; 2026: 29.

257 papers.

Sort:

By area:

  1. 2021The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Gamescited by 903
  2. 2025LMM-R1: Empowering 3B LMMs with Strong Reasoning Abilities Through Two-Stage Rule-Based RLcited by 244
  3. 2023Habitat 3.0: A Co-Habitat for Humans, Avatars and Robotscited by 220
  4. 2022Multi-Agent Reinforcement Learning is a Sequence Modeling Problemcited by 153
  5. 2022SMACv2: An Improved Benchmark for Cooperative Multi-Agent Reinforcement Learningcited by 118
  6. 2022A Survey of Multi-Agent Deep Reinforcement Learning with Communicationcited by 103
  7. 2021Celebrating Diversity in Shared Multi-Agent Reinforcement Learningcited by 96
  8. 2021From Motor Control to Team Play in Simulated Humanoid Footballcited by 87
  9. 2023Heterogeneous-Agent Reinforcement Learningcited by 67
  10. 2022VMAS: A Vectorized Multi-Agent Simulator for Collective Robot Learningcited by 63
  11. 2023TorchRL: A data-driven decision-making library for PyTorchcited by 59
  12. 2022Nocturne: a scalable driving benchmark for bringing multi-agent learning one step closer to the real worldcited by 56
38 more
  1. 2024Learning Multi-Agent Communication from Graph Modeling Perspectivecited by 47
  2. 2021Modelling Behavioural Diversity for Learning in Open-Ended Gamescited by 39
  3. 2023Counterfactual Conservative Q Learning for Offline Multi-agent Reinforcement Learningcited by 36
  4. 2022On Transforming Reinforcement Learning by Transformer: The Development Trajectorycited by 35
  5. 2023Heterogeneous Multi-Robot Reinforcement Learningcited by 34
  6. 2024SoccerNet Game State Reconstruction: End-to-End Athlete Tracking and Identification on a Minimapcited by 34
  7. 2021Learning to Simulate Self-Driven Particles System with Coordinated Policy Optimizationcited by 33
  8. 2022Revisiting Some Common Practices in Cooperative Multi-Agent Reinforcement Learningcited by 32
  9. 2023Multi-agent Reinforcement Learning: A Comprehensive Surveycited by 30
  10. 2022Reward Gaming in Conditional Text Generationcited by 29
  11. 2021Coach-Player Multi-Agent Reinforcement Learning for Dynamic Team Compositioncited by 28
  12. 2021JueWu-MC: Playing Minecraft with Sample-efficient Hierarchical Reinforcement Learningcited by 28
  13. 2022Honor of Kings Arena: an Environment for Generalization in Competitive Reinforcement Learningcited by 27
  14. 2023A Review of Cooperation in Multi-agent Learningcited by 27
  15. 2024A Survey on Self-play Methods in Reinforcement Learningcited by 26
  16. 2021Diverse Auto-Curriculum is Critical for Successful Real-World Multiagent Learning Systemscited by 24
  17. 2023Order Matters: Agent-by-agent Policy Optimizationcited by 24
  18. 2021MALib: A Parallel Framework for Population-based Multi-agent Reinforcement Learningcited by 23
  19. 2023ZSC-Eval: An Evaluation Toolkit and Benchmark for Multi-agent Zero-shot Coordinationcited by 23
  20. 2024Can VLMs Play Action Role-Playing Games? Take Black Myth Wukong as a Study Casecited by 23
  21. 2021Model-Based Opponent Modelingcited by 21
  22. 2022MARLlib: A Scalable and Efficient Multi-agent Reinforcement Learning Librarycited by 21
  23. 2021GridToPix: Training Embodied Agents with Minimal Supervisioncited by 20
  24. 2024POGEMA: A Benchmark Platform for Cooperative Multi-Agent Pathfindingcited by 20
  25. 2021HAVEN: Hierarchical Cooperative Multi-Agent Reinforcement Learning with Dual Coordination Mechanismcited by 19
  26. 2022Evaluation of creating scoring opportunities for teammates in soccer via trajectory predictioncited by 19
  27. 2023Maximum Entropy Heterogeneous-Agent Reinforcement Learningcited by 19
  28. 2021Semantic Tracklets: An Object-Centric Representation for Visual Multi-Agent Reinforcement Learningcited by 18
  29. 2021TiKick: Towards Playing Multi-agent Football Full Games from Single-agent Demonstrationscited by 18
  30. 2023TiZero: Mastering Multi-Agent Football with Curriculum Learning and Self-Playcited by 18
  31. 2023Is Centralized Training with Decentralized Execution Framework Centralized Enough for MARL?cited by 18
  32. 2022Model-based Multi-agent Reinforcement Learning: Recent Progress and Prospectscited by 17
  33. 2022A2C is a special case of PPOcited by 16
  34. 2022LDSA: Learning Dynamic Subtask Assignment in Cooperative Multi-Agent Reinforcement Learningcited by 15
  35. 2022ACE: Cooperative Multi-agent Q-learning with Bidirectional Action-Dependencycited by 15
  36. 2021Unifying Behavioral and Response Diversity for Open-ended Learning in Zero-sum Gamescited by 14
  37. 2023Games for Artificial Intelligence Research: A Review and Perspectivescited by 14
  38. 2023Adaptive Coordination in Social Embodied Rearrangementcited by 14

50 most cited of 257 All 257 (JSON)