mlsanityData to 14 Oct 2026

Towards optimized actions in critical situations of soccer games with deep reinforcement learning

arXiv 2109.06625 ↗First posted 14 Sep 2021cs.LGcited by 1

Not on the map. Why?

Abstract (arXiv)

Soccer is a sparse rewarding game: any smart or careless action in critical situations can change the result of the match. Therefore players, coaches, and scouts are all curious about the best action to be performed in critical situations, such as the times with a high probability of losing ball possession or scoring a goal. This work proposes a new state representation for the soccer game and a batch reinforcement learning to train a smart policy network. This network gets the contextual information of the situation and proposes the optimal action to maximize the expected goal for the team. We performed extensive numerical experiments on the soccer logs made by InStat for 104 European soccer matches. The results show that in all 104 games, the optimized policy obtains higher rewards than its counterpart in the behavior policy. Besides, our framework learns policies that are close to the expected behavior in the real world. For instance, in the optimized policy, we observe that some actions such as foul, or ball out can be sometimes more rewarding than a shot in specific situations.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 9 of its 23 references that are in the map. First posted on arXiv 14 Sep 2021.

Builds on

9 of 23 references.

References · 9

  1. 2014Long-term Recurrent Convolutional Networks for Visual Recognition and Descriptioncited by 484
  2. 2015Doubly Robust Off-policy Value Evaluation for Reinforcement Learningcited by 339
  3. 2018More Robust Doubly Robust Off-policy Evaluationcited by 141
  4. 2019Towards Optimal Off-Policy Evaluation for Reinforcement Learning with Marginalized Importance Samplingcited by 119
  5. 2018Actions Speak Louder Than Goals: Valuing Player Actions in Soccercited by 90
  6. 2018Deep Reinforcement Learning in Ice Hockey for Context-Aware Player Evaluationcited by 23
  7. 2020SoccerMap: A Deep Learning Architecture for Visually-Interpretable Analysis in Soccercited by 17
  8. 2020Cracking the Black Box: Distilling Deep Sports Analyticscited by 12
  9. 2016QPass: a Merit-based Evaluation of Soccer Passescited by 4

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025OpenSTARLab: Open Approach for Spatio-Temporal Agent Data Analysis in Soccercited by 7

Similar, not cited

Close in wording, no citation either way

  1. 2023Action valuation of on- and off-ball soccer players based on multi-agent deep reinforcement learning
  2. 2025Expandable Decision-Making States for Multi-Agent Deep Reinforcement Learning in Soccer Tactical Analysis
  3. 2026Maximising the Set-Piece Return: Optimising Football Corner Tactics with Graph Reinforcement Learning
  4. 2024MARLadona -- Towards Cooperative Team Play Using Multi-Agent Reinforcement Learning
  5. 2025Multi-Agent Reinforcement Learning and Real-Time Decision-Making in Robotic Soccer for Virtual Environments
  6. 2025SoccerDiffusion: Toward Learning End-to-End Humanoid Robot Soccer from Gameplay Recordings
  7. 2025Human-Like Goalkeeping in a Realistic Football Simulation: a Sample-Efficient Reinforcement Learning Approach
  8. 2024Learning Robot Soccer from Egocentric Vision with Deep Reinforcement Learning