# Action valuation of on- and off-ball soccer players based on multi-agent deep reinforcement learning

arXiv: https://arxiv.org/abs/2305.17886 · HTML: https://mlsanity.com/paper/2305.17886/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 47 references that are in the map.
- First posted on arXiv 29 May 2023.

## Builds on (most cited first, 16 of 16)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Google Research Football: A Novel Reinforcement Learning Environment](https://mlsanity.com/paper/1907.11180/) (2019)
- [Actions Speak Louder Than Goals: Valuing Player Actions in Soccer](https://mlsanity.com/paper/1802.07127/) (2018)
- [Coordinated Multi-Agent Imitation Learning](https://mlsanity.com/paper/1703.03121/) (2017)
- [Generating Multi-Agent Trajectories using Programmatic Weak Supervision](https://mlsanity.com/paper/1803.07612/) (2018)
- [Deep Reinforcement Learning in Ice Hockey for Context-Aware Player Evaluation](https://mlsanity.com/paper/1805.11088/) (2018)
- [Evaluation of creating scoring opportunities for teammates in soccer via trajectory prediction](https://mlsanity.com/paper/2206.01899/) (2022)
- [Evaluation of soccer team defense based on prediction models of ball recovery and being attacked: A pilot study](https://mlsanity.com/paper/2103.09627/) (2021)
- [Estimating counterfactual treatment outcomes over time in complex multiagent scenarios](https://mlsanity.com/paper/2206.01900/) (2022)
- [Decentralized policy learning with partial observation and mechanical constraints for multiperson modeling](https://mlsanity.com/paper/2007.03155/) (2020)
- [Transformer-Based Neural Marked Spatio Temporal Point Process Model for Football Match Events Analysis](https://mlsanity.com/paper/2302.09276/) (2023)
- [Adaptive action supervision in reinforcement learning from real-world multi-agent demonstrations](https://mlsanity.com/paper/2305.13030/) (2023)
- [Location analysis of players in UEFA EURO 2020 and 2022 using generalized valuation of defense by estimating probabilities](https://mlsanity.com/paper/2212.00021/) (2022)
- [Data-driven Analysis for Understanding Team Sports Behaviors](https://mlsanity.com/paper/2102.07545/) (2021)
- [How does AI play football? An analysis of RL and real-world football strategies](https://mlsanity.com/paper/2111.12340/) (2021)
- [Measuring Football Players' On-the-ball Contributions From Passes During Games](https://mlsanity.com/paper/1810.02252/) (2018)

## Built on by (most cited first, 10 of 10)

- [OpenSTARLab: Open Approach for Spatio-Temporal Agent Data Analysis in Soccer](https://mlsanity.com/paper/2502.02785/) (2025)
- [TrackID3x3: A Dataset and Algorithm for Multi-Player Tracking with Identification and Pose Estimation in 3x3 Basketball Full-court Videos](https://mlsanity.com/paper/2503.18282/) (2025)
- [Action Valuation in Sports: A Survey](https://mlsanity.com/paper/2504.06163/) (2025)
- [Interpretable Low-Dimensional Modeling of Spatiotemporal Agent States for Decision Making in Football Tactics](https://mlsanity.com/paper/2506.16696/) (2025)
- [Evaluating Movement Initiation Timing in Ultimate Frisbee via Temporal Counterfactuals](https://mlsanity.com/paper/2508.17611/) (2025)
- [Expandable Decision-Making States for Multi-Agent Deep Reinforcement Learning in Soccer Tactical Analysis](https://mlsanity.com/paper/2510.00480/) (2025)
- [Data-driven simulator of multi-animal behavior with unknown dynamics via offline and online reinforcement learning](https://mlsanity.com/paper/2510.10451/) (2025)
- [Maximising the Set-Piece Return: Optimising Football Corner Tactics with Graph Reinforcement Learning](https://mlsanity.com/paper/2606.06353/) (2026)
- [R2D-RL: A RoboCup 2D Soccer Environment for Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2606.18786/) (2026)
- [Strategic Multi-Agent Learning for Interpretable Action Valuation of All Players in Football](https://mlsanity.com/paper/2610.05961/) (2026)

## Abstract (arXiv)

Analysis of invasive sports such as soccer is challenging because the game situation changes continuously in time and space, and multiple agents individually recognize the game situation and make decisions. Previous studies using deep reinforcement learning have often considered teams as a single agent and valued the teams and players who hold the ball in each discrete event. Then it was challenging to value the actions of multiple players, including players far from the ball, in a spatiotemporally continuous state space. In this paper, we propose a method of valuing possible actions for on- and off-ball soccer players in a single holistic framework based on multi-agent deep reinforcement learning. We consider a discrete action space in a continuous state space that mimics that of Google research football and leverages supervised learning for actions in reinforcement learning. In the experiment, we analyzed the relationships with conventional indicators, season goals, and game ratings by experts, and showed the effectiveness of the proposed method. Our approach can assess how multiple players move continuously throughout the game, which is difficult to be discretized or labeled but vital for teamwork, scouting, and fan engagement.