# Deep Reinforcement Learning in Ice Hockey for Context-Aware Player Evaluation

arXiv: https://arxiv.org/abs/1805.11088 · HTML: https://mlsanity.com/paper/1805.11088/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 May 2018.

## Built on by (most cited first, 23 of 23)

- [ShuttleNet: Position-aware Fusion of Rally Progress and Player Styles for Stroke Forecasting in Badminton](https://mlsanity.com/paper/2112.01044/) (2021)
- [Evaluation of creating scoring opportunities for teammates in soccer via trajectory prediction](https://mlsanity.com/paper/2206.01899/) (2022)
- [Estimating counterfactual treatment outcomes over time in complex multiagent scenarios](https://mlsanity.com/paper/2206.01900/) (2022)
- [A Survey of Deep Learning in Sports Applications: Perception, Comprehension, and Decision](https://mlsanity.com/paper/2307.03353/) (2023)
- [Action valuation of on- and off-ball soccer players based on multi-agent deep reinforcement learning](https://mlsanity.com/paper/2305.17886/) (2023)
- [A Comprehensive Survey on Inverse Constrained Reinforcement Learning: Definitions, Progress and Challenges](https://mlsanity.com/paper/2409.07569/) (2024)
- [Adaptive action supervision in reinforcement learning from real-world multi-agent demonstrations](https://mlsanity.com/paper/2305.13030/) (2023)
- [A Strategic Framework for Optimal Decisions in Football 1-vs-1 Shot-Taking Situations: An Integrated Approach of Machine Learning, Theory-Based Modeling, and Game Theory](https://mlsanity.com/paper/2307.14732/) (2023)
- [OpenSTARLab: Open Approach for Spatio-Temporal Agent Data Analysis in Soccer](https://mlsanity.com/paper/2502.02785/) (2025)
- [Data-driven Analysis for Understanding Team Sports Behaviors](https://mlsanity.com/paper/2102.07545/) (2021)
- [Action Valuation in Sports: A Survey](https://mlsanity.com/paper/2504.06163/) (2025)
- [Development of an expected possession value model to analyse team attacking performances in rugby league](https://mlsanity.com/paper/2105.05303/) (2021)
- [Towards optimized actions in critical situations of soccer games with deep reinforcement learning](https://mlsanity.com/paper/2109.06625/) (2021)
- [A Multi-stage deep architecture for summary generation of soccer videos](https://mlsanity.com/paper/2205.00694/) (2022)
- [Performance Insights-based AI-driven Football Transfer Fee Prediction](https://mlsanity.com/paper/2401.16795/) (2024)
- [Expandable Decision-Making States for Multi-Agent Deep Reinforcement Learning in Soccer Tactical Analysis](https://mlsanity.com/paper/2510.00480/) (2025)
- [Bandit Modeling of Map Selection in Counter-Strike: Global Offensive](https://mlsanity.com/paper/2106.08888/) (2021)
- [Optimal Team Economic Decisions in Counter-Strike](https://mlsanity.com/paper/2109.12990/) (2021)
- [CAMP: A Context-Aware Cricket Players Performance Metric](https://mlsanity.com/paper/2307.13700/) (2023)
- [Forecasting NCAA Basketball Outcomes with Deep Learning: A Comparative Study of LSTM and Transformer Models](https://mlsanity.com/paper/2508.02725/) (2025)
- [Gaining Momentum: Uncovering Hidden Scoring Dynamics in Hockey through Deep Neural Sequencing and Causal Modeling](https://mlsanity.com/paper/2511.00615/) (2025)
- [H-VAEP and H-xT: Valuing Offensive On-the-Ball Actions in Handball by Estimating Probabilities](https://mlsanity.com/paper/2608.12926/) (2026)
- [Strategic Multi-Agent Learning for Interpretable Action Valuation of All Players in Football](https://mlsanity.com/paper/2610.05961/) (2026)

## Abstract (arXiv)

A variety of machine learning models have been proposed to assess the performance of players in professional sports. However, they have only a limited ability to model how player performance depends on the game context. This paper proposes a new approach to capturing game context: we apply Deep Reinforcement Learning (DRL) to learn an action-value Q function from 3M play-by-play events in the National Hockey League (NHL). The neural network representation integrates both continuous context signals and game history, using a possession-based LSTM. The learned Q-function is used to value players' actions under different game contexts. To assess a player's overall performance, we introduce a novel Game Impact Metric (GIM) that aggregates the values of the player's actions. Empirical Evaluation shows GIM is consistent throughout a play season, and correlates highly with standard success measures and future salary.