mlsanityData to 14 Oct 2026

Strategic Multi-Agent Learning for Interpretable Action Valuation of All Players in Football

arXiv 2610.05961 ↗First posted 5 Oct 2026cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Valuing player actions in football requires accounting for strategic interactions among 22 players, including off-ball movements and defensive positioning. Existing reinforcement-learning-based methods commonly aggregate decisions at the team level or estimate player values independently, leaving strategic interdependence among players insufficiently represented. This study proposes an action valuation framework inspired by Markov perfect equilibrium (MPE) for all players. Each possession is modeled as a finite-horizon dynamic game, with each player represented as an autonomous agent whose policy depends on the current game state. MPE is used as a motivating solution concept rather than an exact equilibrium. To improve interpretability, we use Expandable Decision-Making States (EDMS) and decompose the Q-value into a successor-feature basis and a linear reward-weight vector. The value basis is estimated by linear TD initialization followed by nonlinear refinement. Using tracking and event data from 95 J1 League matches, we compare the proposed formulation with an independent reinforcement learning baseline. Because the two formulations define TD errors in different target spaces, TD MSE is used only for within-formulation consistency. With EDMS fixed, the independent baseline assigns the highest value to forward movement in 99.21% of evaluated off-ball states, whereas the most frequent direction under the proposed formulation accounts for 17.63%. Team-level average Q-values show a negative association with season-level expected goals for the baseline and a weakly positive association for the proposed formulation. Qualitative analyses illustrate context-dependent valuations of off-ball movements and defensive positioning. Overall, the proposed formulation produces more context-sensitive action rankings, although the comparison does not isolate the MPE-inspired component.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 16 of its 42 references that are in the map. First posted on arXiv 5 Oct 2026.

Builds on

16 of 42 references.

References · 16

  1. 2019Google Research Football: A Novel Reinforcement Learning Environmentcited by 260
  2. 2017Coordinated Multi-Agent Imitation Learningcited by 72
  3. 2023TacticAI: an AI assistant for football tacticscited by 44
  4. 2018Deep Reinforcement Learning in Ice Hockey for Context-Aware Player Evaluationcited by 23
  5. 2022Evaluation of creating scoring opportunities for teammates in soccer via trajectory predictioncited by 20
  6. 2023TiZero: Mastering Multi-Agent Football with Curriculum Learning and Self-Playcited by 20
  7. 2021Evaluation of soccer team defense based on prediction models of ball recovery and being attacked: A pilot studycited by 17
  8. 2022Estimating counterfactual treatment outcomes over time in complex multiagent scenarioscited by 12
  9. 2020Decentralized policy learning with partial observation and mechanical constraints for multiperson modelingcited by 10
  10. 2023Action valuation of on- and off-ball soccer players based on multi-agent deep reinforcement learningcited by 10
  11. 2020Optimising Game Tactics for Footballcited by 9
  12. 2023Adaptive action supervision in reinforcement learning from real-world multi-agent demonstrationscited by 9
4 more
  1. 2022Location analysis of players in UEFA EURO 2020 and 2022 using generalized valuation of defense by estimating probabilitiescited by 8
  2. 2021How does AI play football? An analysis of RL and real-world football strategiescited by 6
  3. 2025Interpretable Low-Dimensional Modeling of Spatiotemporal Agent States for Decision Making in Football Tacticscited by 2
  4. 2025Expandable Decision-Making States for Multi-Agent Deep Reinforcement Learning in Soccer Tactical Analysiscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Human-Like Goalkeeping in a Realistic Football Simulation: a Sample-Efficient Reinforcement Learning Approach
  2. 2026Maximising the Set-Piece Return: Optimising Football Corner Tactics with Graph Reinforcement Learning
  3. 2025Improving Sample Efficiency in Multi-Agent Reinforcement Learning for Simulated Football Games via Exploration
  4. 2026TacticGen: Grounding Adaptable and Scalable Generation of Football Tactics
  5. 2021From Motor Control to Team Play in Simulated Humanoid Football
  6. 2025Multi-Agent Reinforcement Learning and Real-Time Decision-Making in Robotic Soccer for Virtual Environments
  7. 2026SportD: How do VLMs physically strategize?
  8. 2026Monte Carlo Pass Search: Using Trajectory Generation for 3D Counterfactual Pass Evaluation in Football