# SoccerNet-Tracking: Multiple Object Tracking Dataset and Benchmark in Soccer Videos

arXiv: https://arxiv.org/abs/2204.06918 · HTML: https://mlsanity.com/paper/2204.06918/

## Key facts

- Cited by 80 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 83 references that are in the map.
- First posted on arXiv 14 Apr 2022.

## Builds on (most cited first, 30 of 31)

- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [YOLOX: Exceeding YOLO Series in 2021](https://mlsanity.com/paper/2107.08430/) (2021)
- [Simple Online and Realtime Tracking](https://mlsanity.com/paper/1602.00763/) (2016)
- [ByteTrack: Multi-Object Tracking by Associating Every Detection Box](https://mlsanity.com/paper/2110.06864/) (2021)
- [HOTA: A Higher Order Metric for Evaluating Multi-Object Tracking](https://mlsanity.com/paper/2009.07736/) (2020)
- [MOT16: A Benchmark for Multi-Object Tracking](https://mlsanity.com/paper/1603.00831/) (2016)
- [FairMOT: On the Fairness of Detection and Re-Identification in Multiple Object Tracking](https://mlsanity.com/paper/2004.01888/) (2020)
- [Tracking without bells and whistles](https://mlsanity.com/paper/1903.05625/) (2019)
- [MOT20: A benchmark for multi object tracking in crowded scenes](https://mlsanity.com/paper/2003.09003/) (2020)
- [Towards Real-Time Multi-Object Tracking](https://mlsanity.com/paper/1909.12605/) (2019)
- [CrowdHuman: A Benchmark for Detecting Human in a Crowd](https://mlsanity.com/paper/1805.00123/) (2018)
- [Joint Detection and Identification Feature Learning for Person Search](https://mlsanity.com/paper/1604.01850/) (2016)
- [CityPersons: A Diverse Dataset for Pedestrian Detection](https://mlsanity.com/paper/1702.05693/) (2017)
- [Multiple Object Tracking: A Literature Review](https://mlsanity.com/paper/1409.7618/) (2014)
- [Person Re-identification in the Wild](https://mlsanity.com/paper/1604.02531/) (2016)
- [SoccerNet: A Scalable Dataset for Action Spotting in Soccer Videos](https://mlsanity.com/paper/1804.04527/) (2018)
- [TAO: A Large-Scale Benchmark for Tracking Any Object](https://mlsanity.com/paper/2005.10356/) (2020)
- [MOTChallenge 2015: Towards a Benchmark for Multi-Target Tracking](https://mlsanity.com/paper/1504.01942/) (2015)
- [SoccerNet-v2: A Dataset and Benchmarks for Holistic Understanding of Broadcast Soccer Videos](https://mlsanity.com/paper/2011.13367/) (2020)
- [UA-DETRAC: A New Benchmark and Protocol for Multi-Object Detection and Tracking](https://mlsanity.com/paper/1511.04136/) (2015)
- [Deep Affinity Network for Multiple Object Tracking](https://mlsanity.com/paper/1810.11780/) (2018)
- [A Context-Aware Loss Function for Action Spotting in Soccer Videos](https://mlsanity.com/paper/1912.01326/) (2019)
- [Camera Calibration and Player Localization in SoccerNet-v2 and Investigation of their Representations for Action Spotting](https://mlsanity.com/paper/2104.09333/) (2021)
- [Tracking Pedestrian Heads in Dense Crowd](https://mlsanity.com/paper/2103.13516/) (2021)
- [3D-ZeF: A 3D Zebrafish Tracking Benchmark Dataset](https://mlsanity.com/paper/2006.08466/) (2020)
- [Using Player's Body-Orientation to Model Pass Feasibility in Soccer](https://mlsanity.com/paper/2004.07209/) (2020)
- [Group Activity Detection from Trajectory and Video Data in Soccer](https://mlsanity.com/paper/2004.10299/) (2020)
- [Contrastive Learning for Sports Video: Unsupervised Player Classification](https://mlsanity.com/paper/2104.10068/) (2021)
- [Multimodal and multiview distillation for real-time player detection on a football field](https://mlsanity.com/paper/2004.07544/) (2020)
- [Self-Supervised Small Soccer Player Detection and Tracking](https://mlsanity.com/paper/2011.10336/) (2020)

## Built on by (most cited first, 30 of 80)

- [SportsMOT: A Large Multi-Object Tracking Dataset in Multiple Sports Scenes](https://mlsanity.com/paper/2304.05170/) (2023)
- [Hard to Track Objects with Irregular Motions and Similar Appearances? Make It Easier by Buffering the Matching Space](https://mlsanity.com/paper/2211.14317/) (2022)
- [Single-Model and Any-Modality for Video Object Tracking](https://mlsanity.com/paper/2311.15851/) (2023)
- [SoccerNet-Caption: Dense Video Captioning for Soccer Broadcasts Commentaries](https://mlsanity.com/paper/2304.04565/) (2023)
- [VARS: Video Assistant Referee System for Automated Soccer Decision Making from Multiple Views](https://mlsanity.com/paper/2304.04617/) (2023)
- [DeepSportradar-v1: Computer Vision Dataset for Sports Understanding with High Quality Annotations](https://mlsanity.com/paper/2208.08190/) (2022)
- [MatchTime: Towards Automatic Soccer Game Commentary Generation](https://mlsanity.com/paper/2406.18530/) (2024)
- [Iterative Scale-Up ExpansionIoU and Deep Features Association for Multi-Object Tracking in Sports](https://mlsanity.com/paper/2306.13074/) (2023)
- [SoccerNet 2022 Challenges Results](https://mlsanity.com/paper/2210.02365/) (2022)
- [X-VARS: Introducing Explainability in Football Refereeing with Multi-Modal Large Language Model](https://mlsanity.com/paper/2404.06332/) (2024)
- [Towards Universal Soccer Video Understanding](https://mlsanity.com/paper/2412.01820/) (2024)
- [SoccerNet 2023 Challenges Results](https://mlsanity.com/paper/2309.06006/) (2023)
- [Towards Active Learning for Action Spotting in Association Football Videos](https://mlsanity.com/paper/2304.04220/) (2023)
- [A Universal Protocol to Benchmark Camera Calibration for Sports](https://mlsanity.com/paper/2404.09807/) (2024)
- [Multi-task Learning for Joint Re-identification, Team Affiliation, and Role Classification for Sports Visual Tracking](https://mlsanity.com/paper/2401.09942/) (2024)
- [Visible-Thermal Multiple Object Tracking: Large-scale Video Dataset and Progressive Fusion Approach](https://mlsanity.com/paper/2408.00969/) (2024)
- [TimeSoccer: An End-to-End Multimodal Large Language Model for Soccer Commentary Generation](https://mlsanity.com/paper/2504.17365/) (2025)
- [SoccerNet-Echoes: A Soccer Game Audio Commentary Dataset](https://mlsanity.com/paper/2405.07354/) (2024)
- [A Survey of Deep Learning in Sports Applications: Perception, Comprehension, and Decision](https://mlsanity.com/paper/2307.03353/) (2023)
- [TeamTrack: A Dataset for Multi-Sport Multi-Object Tracking in Full-pitch Videos](https://mlsanity.com/paper/2404.13868/) (2024)
- [SoccerMaster: A Vision Foundation Model for Soccer Understanding](https://mlsanity.com/paper/2512.11016/) (2025)
- [Location analysis of players in UEFA EURO 2020 and 2022 using generalized valuation of defense by estimating probabilities](https://mlsanity.com/paper/2212.00021/) (2022)
- [All Keypoints You Need: Detecting Arbitrary Keypoints on the Body of Triple, High, and Long Jump Athletes](https://mlsanity.com/paper/2304.02939/) (2023)
- [Towards AI-Powered Video Assistant Referee System (VARS) for Association Football](https://mlsanity.com/paper/2407.12483/) (2024)
- [SoccerNet 2024 Challenges Results](https://mlsanity.com/paper/2409.10587/) (2024)
- [SoccerChat: Integrating Multimodal Data for Enhanced Soccer Game Understanding](https://mlsanity.com/paper/2505.16630/) (2025)
- [CLIP-ReIdent: Contrastive Training for Player Re-Identification](https://mlsanity.com/paper/2303.11855/) (2023)
- [TAO-Amodal: A Benchmark for Tracking Any Object Amodally](https://mlsanity.com/paper/2312.12433/) (2023)
- [Foul prediction with estimated poses from soccer broadcast video](https://mlsanity.com/paper/2402.09650/) (2024)
- [Deep learning for action spotting in association football videos](https://mlsanity.com/paper/2410.01304/) (2024)

## Abstract (arXiv)

Tracking objects in soccer videos is extremely important to gather both player and team statistics, whether it is to estimate the total distance run, the ball possession or the team formation. Video processing can help automating the extraction of those information, without the need of any invasive sensor, hence applicable to any team on any stadium. Yet, the availability of datasets to train learnable models and benchmarks to evaluate methods on a common testbed is very limited. In this work, we propose a novel dataset for multiple object tracking composed of 200 sequences of 30s each, representative of challenging soccer scenarios, and a complete 45-minutes half-time for long-term tracking. The dataset is fully annotated with bounding boxes and tracklet IDs, enabling the training of MOT baselines in the soccer domain and a full benchmarking of those methods on our segregated challenge sets. Our analysis shows that multiple player, referee and ball tracking in soccer videos is far from being solved, with several improvement required in case of fast motion or in scenarios of severe occlusion.