# SoccerNet Game State Reconstruction: End-to-End Athlete Tracking and Identification on a Minimap

arXiv: https://arxiv.org/abs/2404.11335 · HTML: https://mlsanity.com/paper/2404.11335/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- Builds on 51 of its 102 references that are in the map.
- First posted on arXiv 17 Apr 2024.

## Builds on (most cited first, 30 of 51)

- [Simple Online and Realtime Tracking with a Deep Association Metric](https://mlsanity.com/paper/1703.07402/) (2017)
- [Simple Online and Realtime Tracking](https://mlsanity.com/paper/1602.00763/) (2016)
- [ByteTrack: Multi-Object Tracking by Associating Every Detection Box](https://mlsanity.com/paper/2110.06864/) (2021)
- [HOTA: A Higher Order Metric for Evaluating Multi-Object Tracking](https://mlsanity.com/paper/2009.07736/) (2020)
- [FairMOT: On the Fairness of Detection and Re-Identification in Multiple Object Tracking](https://mlsanity.com/paper/2004.01888/) (2020)
- [Tracking without bells and whistles](https://mlsanity.com/paper/1903.05625/) (2019)
- [Google Research Football: A Novel Reinforcement Learning Environment](https://mlsanity.com/paper/1907.11180/) (2019)
- [Real-time Scene Text Detection with Differentiable Binarization](https://mlsanity.com/paper/1911.08947/) (2019)
- [StrongSORT: Make DeepSORT Great Again](https://mlsanity.com/paper/2202.13514/) (2022)
- [SoccerNet-v2: A Dataset and Benchmarks for Holistic Understanding of Broadcast Soccer Videos](https://mlsanity.com/paper/2011.13367/) (2020)
- [Show, Attend and Read: A Simple and Strong Baseline for Irregular Text Recognition](https://mlsanity.com/paper/1811.00751/) (2018)
- [Deep Affinity Network for Multiple Object Tracking](https://mlsanity.com/paper/1810.11780/) (2018)
- [Body Part-Based Representation Learning for Occluded Person Re-Identification](https://mlsanity.com/paper/2211.03679/) (2022)
- [SoccerNet-Caption: Dense Video Captioning for Soccer Broadcasts Commentaries](https://mlsanity.com/paper/2304.04565/) (2023)
- [A Context-Aware Loss Function for Action Spotting in Soccer Videos](https://mlsanity.com/paper/1912.01326/) (2019)
- [Camera Calibration and Player Localization in SoccerNet-v2 and Investigation of their Representations for Action Spotting](https://mlsanity.com/paper/2104.09333/) (2021)
- [MMOCR: A Comprehensive Toolbox for Text Detection, Recognition and Understanding](https://mlsanity.com/paper/2108.06543/) (2021)
- [Spotting Temporally Precise, Fine-Grained Events in Video](https://mlsanity.com/paper/2207.10213/) (2022)
- [VARS: Video Assistant Referee System for Automated Soccer Decision Making from Multiple Views](https://mlsanity.com/paper/2304.04617/) (2023)
- [DeepSportradar-v1: Computer Vision Dataset for Sports Understanding with High Quality Annotations](https://mlsanity.com/paper/2208.08190/) (2022)
- [Temporally-Aware Feature Pooling for Action Spotting in Soccer Broadcasts](https://mlsanity.com/paper/2104.06779/) (2021)
- [Iterative Scale-Up ExpansionIoU and Deep Features Association for Multi-Object Tracking in Sports](https://mlsanity.com/paper/2306.13074/) (2023)
- [SoccerNet 2022 Challenges Results](https://mlsanity.com/paper/2210.02365/) (2022)
- [Feature Combination Meets Attention: Baidu Soccer Embeddings and Transformer based Temporal Detection](https://mlsanity.com/paper/2106.14447/) (2021)
- [Temporally Precise Action Spotting in Soccer Videos Using Dense Detection Anchors](https://mlsanity.com/paper/2205.10450/) (2022)
- [Semi-Supervised Training to Improve Player and Ball Detection in Soccer](https://mlsanity.com/paper/2204.06859/) (2022)
- [TVCalib: Camera Calibration for Sports Field Registration in Soccer](https://mlsanity.com/paper/2207.11709/) (2022)
- [X-VARS: Introducing Explainability in Football Refereeing with Multi-Modal Large Language Model](https://mlsanity.com/paper/2404.06332/) (2024)
- [A Comprehensive Review of Computer Vision in Sports: Open Issues, Future Trends and Research Directions](https://mlsanity.com/paper/2203.02281/) (2022)
- [Using Player's Body-Orientation to Model Pass Feasibility in Soccer](https://mlsanity.com/paper/2004.07209/) (2020)

## Built on by (most cited first, 30 of 34)

- [Towards Universal Soccer Video Understanding](https://mlsanity.com/paper/2412.01820/) (2024)
- [Multi-Agent System for Comprehensive Soccer Understanding](https://mlsanity.com/paper/2505.03735/) (2025)
- [Keypoint Promptable Re-Identification](https://mlsanity.com/paper/2407.18112/) (2024)
- [SoccerNet-Echoes: A Soccer Game Audio Commentary Dataset](https://mlsanity.com/paper/2405.07354/) (2024)
- [SoccerMaster: A Vision Foundation Model for Soccer Understanding](https://mlsanity.com/paper/2512.11016/) (2025)
- [Towards AI-Powered Video Assistant Referee System (VARS) for Association Football](https://mlsanity.com/paper/2407.12483/) (2024)
- [SoccerNet 2024 Challenges Results](https://mlsanity.com/paper/2409.10587/) (2024)
- [Deep learning for action spotting in association football videos](https://mlsanity.com/paper/2410.01304/) (2024)
- [BroadTrack: Broadcast Camera Tracking for Soccer](https://mlsanity.com/paper/2412.01721/) (2024)
- [TrackID3x3: A Dataset and Algorithm for Multi-Player Tracking with Identification and Pose Estimation in 3x3 Basketball Full-court Videos](https://mlsanity.com/paper/2503.18282/) (2025)
- [CAMELTrack: Context-Aware Multi-cue ExpLoitation for Online Multi-Object Tracking](https://mlsanity.com/paper/2505.01257/) (2025)
- [SoccerNet 2025 Challenges Results](https://mlsanity.com/paper/2508.19182/) (2025)
- [GenTac: Generative Modeling and Forecasting of Soccer Tactics](https://mlsanity.com/paper/2604.11786/) (2026)
- [OSL-ActionSpotting: A Unified Library for Action Spotting in Sports Videos](https://mlsanity.com/paper/2407.01265/) (2024)
- [SoccerSynth-Detection: A Synthetic Dataset for Soccer Player Detection](https://mlsanity.com/paper/2501.09281/) (2025)
- [Action Valuation in Sports: A Survey](https://mlsanity.com/paper/2504.06163/) (2025)
- [Can Geometry Save Central Views for Sports Field Registration?](https://mlsanity.com/paper/2504.20052/) (2025)
- [Pixels or Positions? Benchmarking Modalities in Group Activity Recognition](https://mlsanity.com/paper/2511.12606/) (2025)
- [From Broadcast to Minimap: Achieving State-of-the-Art SoccerNet Game State Reconstruction](https://mlsanity.com/paper/2504.06357/) (2025)
- [Expandable Decision-Making States for Multi-Agent Deep Reinforcement Learning in Soccer Tactical Analysis](https://mlsanity.com/paper/2510.00480/) (2025)
- [Multi-Modal Soccer Scene Analysis with Masked Pre-Training](https://mlsanity.com/paper/2512.19528/) (2025)
- [Towards Athlete Fatigue Assessment from Association Football Videos](https://mlsanity.com/paper/2604.05636/) (2026)
- [SoccerNet 2026 Challenges Results](https://mlsanity.com/paper/2607.07320/) (2026)
- [Training-Free Off-Screen Player Imputation for Broadcast-Based Spatial Football Analytics](https://mlsanity.com/paper/2607.11548/) (2026)
- [Space-Creating versus Dead Possession: An Off-Ball Possession-Quality Index for Broadcast Football](https://mlsanity.com/paper/2608.09887/) (2026)
- [Improving Weakly-supervised Video Instance Segmentation by Leveraging Spatio-temporal Consistency](https://mlsanity.com/paper/2408.16661/) (2024)
- [SoccerTrack v2: A Full-Pitch Panoramic Video Dataset for Game State Reconstruction and Ball Action Spotting](https://mlsanity.com/paper/2508.01802/) (2025)
- [TacSIm: A Dataset and Benchmark for Football Tactical Style Imitation](https://mlsanity.com/paper/2603.25199/) (2026)
- [SoccerRef-Agents: Multi-Agent System for Automated Soccer Refereeing](https://mlsanity.com/paper/2604.23392/) (2026)
- [BasketEvent: Understanding Who Did What and When in Basketball Videos](https://mlsanity.com/paper/2607.21267/) (2026)

## Abstract (arXiv)

Tracking and identifying athletes on the pitch holds a central role in collecting essential insights from the game, such as estimating the total distance covered by players or understanding team tactics. This tracking and identification process is crucial for reconstructing the game state, defined by the athletes' positions and identities on a 2D top-view of the pitch, (i.e. a minimap). However, reconstructing the game state from videos captured by a single camera is challenging. It requires understanding the position of the athletes and the viewpoint of the camera to localize and identify players within the field. In this work, we formalize the task of Game State Reconstruction and introduce SoccerNet-GSR, a novel Game State Reconstruction dataset focusing on football videos. SoccerNet-GSR is composed of 200 video sequences of 30 seconds, annotated with 9.37 million line points for pitch localization and camera calibration, as well as over 2.36 million athlete positions on the pitch with their respective role, team, and jersey number. Furthermore, we introduce GS-HOTA, a novel metric to evaluate game state reconstruction methods. Finally, we propose and release an end-to-end baseline for game state reconstruction, bootstrapping the research on this task. Our experiments show that GSR is a challenging novel task, which opens the field for future research. Our dataset and codebase are publicly available at https://github.com/SoccerNet/sn-gamestate.