# Contrastive Learning for Sports Video: Unsupervised Player Classification

arXiv: https://arxiv.org/abs/2104.10068 · HTML: https://mlsanity.com/paper/2104.10068/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 27 references that are in the map.
- First posted on arXiv 15 Apr 2021.

## Builds on (most cited first, 10 of 10)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [YOLACT: Real-time Instance Segmentation](https://mlsanity.com/paper/1904.02689/) (2019)
- [Deep metric learning using Triplet network](https://mlsanity.com/paper/1412.6622/) (2014)
- [Towards K-means-friendly Spaces: Simultaneous Deep Learning and Clustering](https://mlsanity.com/paper/1610.04794/) (2016)
- [Joint Unsupervised Learning of Deep Representations and Image Clusters](https://mlsanity.com/paper/1604.03628/) (2016)
- [SoccerNet: A Scalable Dataset for Action Spotting in Soccer Videos](https://mlsanity.com/paper/1804.04527/) (2018)

## Built on by (most cited first, 17 of 17)

- [SoccerNet-Tracking: Multiple Object Tracking Dataset and Benchmark in Soccer Videos](https://mlsanity.com/paper/2204.06918/) (2022)
- [Sports-QA: A Large-Scale Video Question Answering Benchmark for Complex and Professional Sports](https://mlsanity.com/paper/2401.01505/) (2024)
- [A Survey on Video Action Recognition in Sports: Datasets, Methods and Applications](https://mlsanity.com/paper/2206.01038/) (2022)
- [Player Tracking and Identification in Ice Hockey](https://mlsanity.com/paper/2110.03090/) (2021)
- [COMPOSER: Compositional Reasoning of Group Activity in Videos with Keypoint-Only Modality](https://mlsanity.com/paper/2112.05892/) (2021)
- [Multi-task Learning for Joint Re-identification, Team Affiliation, and Role Classification for Sports Visual Tracking](https://mlsanity.com/paper/2401.09942/) (2024)
- [A Survey of Deep Learning in Sports Applications: Perception, Comprehension, and Decision](https://mlsanity.com/paper/2307.03353/) (2023)
- [A Graph-Based Method for Soccer Action Spotting Using Unsupervised Player Classification](https://mlsanity.com/paper/2211.12334/) (2022)
- [Tracking Skiers from the Top to the Bottom](https://mlsanity.com/paper/2312.09723/) (2023)
- [Extraction of Positional Player Data from Broadcast Soccer Videos](https://mlsanity.com/paper/2110.11107/) (2021)
- [P2ANet: A Dataset and Benchmark for Dense Action Detection from Table Tennis Match Broadcasting Videos](https://mlsanity.com/paper/2207.12730/) (2022)
- [TrackID3x3: A Dataset and Algorithm for Multi-Player Tracking with Identification and Pose Estimation in 3x3 Basketball Full-court Videos](https://mlsanity.com/paper/2503.18282/) (2025)
- [Weakly-supervised Semantic Segmentation via Dual-stream Contrastive Learning of Cross-image Contextual Information](https://mlsanity.com/paper/2405.04913/) (2024)
- [Runner re-identification from single-view running video in the open-world setting](https://mlsanity.com/paper/2310.11700/) (2023)
- [Towards long-term player tracking with graph hierarchies and domain-specific features](https://mlsanity.com/paper/2502.21242/) (2025)
- [Capsule Network based Contrastive Learning of Unsupervised Visual Representations](https://mlsanity.com/paper/2209.11276/) (2022)
- [Calisthenics Skills Temporal Video Segmentation](https://mlsanity.com/paper/2507.12245/) (2025)

## Abstract (arXiv)

We address the problem of unsupervised classification of players in a team sport according to their team affiliation, when jersey colours and design are not known a priori. We adopt a contrastive learning approach in which an embedding network learns to maximize the distance between representations of players on different teams relative to players on the same team, in a purely unsupervised fashion, without any labelled data. We evaluate the approach using a new hockey dataset and find that it outperforms prior unsupervised approaches by a substantial margin, particularly for real-time application when only a small number of frames are available for unsupervised learning before team assignments must be made. Remarkably, we show that our contrastive method achieves 94% accuracy after unsupervised training on only a single frame, with accuracy rising to 97% within 500 frames (17 seconds of game time). We further demonstrate how accurate team classification allows accurate team-conditional heat maps of player positioning to be computed.