# How does AI play football? An analysis of RL and real-world football strategies

arXiv: https://arxiv.org/abs/2111.12340 · HTML: https://mlsanity.com/paper/2111.12340/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 41 references that are in the map.
- First posted on arXiv 24 Nov 2021.

## Builds on (most cited first, 15 of 15)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model](https://mlsanity.com/paper/1911.08265/) (2019)
- [Dota 2 with Large Scale Deep Reinforcement Learning](https://mlsanity.com/paper/1912.06680/) (2019)
- [IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architectures](https://mlsanity.com/paper/1802.01561/) (2018)
- [Distributed Prioritized Experience Replay](https://mlsanity.com/paper/1803.00933/) (2018)
- [Google Research Football: A Novel Reinforcement Learning Environment](https://mlsanity.com/paper/1907.11180/) (2019)
- [Actions Speak Louder Than Goals: Valuing Player Actions in Soccer](https://mlsanity.com/paper/1802.07127/) (2018)
- [From Motor Control to Team Play in Simulated Humanoid Football](https://mlsanity.com/paper/2105.12196/) (2021)
- [Coordinated Multi-Agent Imitation Learning](https://mlsanity.com/paper/1703.03121/) (2017)
- [Emergent Coordination Through Competition](https://mlsanity.com/paper/1902.07151/) (2019)
- [Evaluating Soccer Player: from Live Camera to Deep Reinforcement Learning](https://mlsanity.com/paper/2101.05388/) (2021)
- [Automatic Extraction of the Passing Strategies of Soccer Teams](https://mlsanity.com/paper/1508.02171/) (2015)
- [Reinforcement Learning from Hierarchical Critics](https://mlsanity.com/paper/1902.03079/) (2019)

## Built on by (most cited first, 6 of 6)

- [Evaluation of creating scoring opportunities for teammates in soccer via trajectory prediction](https://mlsanity.com/paper/2206.01899/) (2022)
- [Action valuation of on- and off-ball soccer players based on multi-agent deep reinforcement learning](https://mlsanity.com/paper/2305.17886/) (2023)
- [Adaptive action supervision in reinforcement learning from real-world multi-agent demonstrations](https://mlsanity.com/paper/2305.13030/) (2023)
- [A Strategic Framework for Optimal Decisions in Football 1-vs-1 Shot-Taking Situations: An Integrated Approach of Machine Learning, Theory-Based Modeling, and Game Theory](https://mlsanity.com/paper/2307.14732/) (2023)
- [Expandable Decision-Making States for Multi-Agent Deep Reinforcement Learning in Soccer Tactical Analysis](https://mlsanity.com/paper/2510.00480/) (2025)
- [Strategic Multi-Agent Learning for Interpretable Action Valuation of All Players in Football](https://mlsanity.com/paper/2610.05961/) (2026)

## Abstract (arXiv)

Recent advances in reinforcement learning (RL) have made it possible to develop sophisticated agents that excel in a wide range of applications. Simulations using such agents can provide valuable information in scenarios that are difficult to scientifically experiment in the real world. In this paper, we examine the play-style characteristics of football RL agents and uncover how strategies may develop during training. The learnt strategies are then compared with those of real football players. We explore what can be learnt from the use of simulated environments by using aggregated statistics and social network analysis (SNA). As a result, we found that (1) there are strong correlations between the competitiveness of an agent and various SNA metrics and (2) aspects of the RL agents play style become similar to real world footballers as the agent becomes more competitive. We discuss further advances that may be necessary to improve our understanding necessary to fully utilise RL for the analysis of football.