mlsanityData to 14 Oct 2026

How does AI play football? An analysis of RL and real-world football strategies

arXiv 2111.12340 ↗First posted 24 Nov 2021cs.AI cs.MAcited by 6

Not on the map. Why?

Abstract (arXiv)

Recent advances in reinforcement learning (RL) have made it possible to develop sophisticated agents that excel in a wide range of applications. Simulations using such agents can provide valuable information in scenarios that are difficult to scientifically experiment in the real world. In this paper, we examine the play-style characteristics of football RL agents and uncover how strategies may develop during training. The learnt strategies are then compared with those of real football players. We explore what can be learnt from the use of simulated environments by using aggregated statistics and social network analysis (SNA). As a result, we found that (1) there are strong correlations between the competitiveness of an agent and various SNA metrics and (2) aspects of the RL agents play style become similar to real world footballers as the agent becomes more competitive. We discuss further advances that may be necessary to improve our understanding necessary to fully utilise RL for the analysis of football.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 15 of its 41 references that are in the map. First posted on arXiv 24 Nov 2021.

Builds on

15 of 41 references.

References · 15

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2017Proximal Policy Optimization Algorithmscited by 15,215
  3. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  4. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,353
  5. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,103
  6. 2018IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architecturescited by 883
  7. 2018Distributed Prioritized Experience Replaycited by 269
  8. 2019Google Research Football: A Novel Reinforcement Learning Environmentcited by 260
  9. 2018Actions Speak Louder Than Goals: Valuing Player Actions in Soccercited by 90
  10. 2021From Motor Control to Team Play in Simulated Humanoid Footballcited by 87
  11. 2017Coordinated Multi-Agent Imitation Learningcited by 72
  12. 2019Emergent Coordination Through Competitioncited by 57
3 more
  1. 2021Evaluating Soccer Player: from Live Camera to Deep Reinforcement Learningcited by 4
  2. 2015Automatic Extraction of the Passing Strategies of Soccer Teamscited by 3
  3. 2019Reinforcement Learning from Hierarchical Criticscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 1; 2023: 3; 2024: 0; 2025: 1; 2026: 1.

6 papers.

  1. 2022Evaluation of creating scoring opportunities for teammates in soccer via trajectory predictioncited by 20
  2. 2023Action valuation of on- and off-ball soccer players based on multi-agent deep reinforcement learningcited by 10
  3. 2023Adaptive action supervision in reinforcement learning from real-world multi-agent demonstrationscited by 9
  4. 2023A Strategic Framework for Optimal Decisions in Football 1-vs-1 Shot-Taking Situations: An Integrated Approach of Machine Learning, Theory-Based Modeling, and Game Theorycited by 7
  5. 2025Expandable Decision-Making States for Multi-Agent Deep Reinforcement Learning in Soccer Tactical Analysiscited by 1
  6. 2026Strategic Multi-Agent Learning for Interpretable Action Valuation of All Players in Footballcited by 0