# A Survey on Video Action Recognition in Sports: Datasets, Methods and Applications

arXiv: https://arxiv.org/abs/2206.01038 · HTML: https://mlsanity.com/paper/2206.01038/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 149 of its 370 references that are in the map.
- First posted on arXiv 2 Jun 2022.

## Builds on (most cited first, 30 of 149)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [BEiT: BERT Pre-Training of Image Transformers](https://mlsanity.com/paper/2106.08254/) (2021)
- [Non-local Neural Networks](https://mlsanity.com/paper/1711.07971/) (2017)
- [The Kinetics Human Action Video Dataset](https://mlsanity.com/paper/1705.06950/) (2017)
- [SlowFast Networks for Video Recognition](https://mlsanity.com/paper/1812.03982/) (2018)
- [Is Space-Time Attention All You Need for Video Understanding?](https://mlsanity.com/paper/2102.05095/) (2021)
- [ViViT: A Video Vision Transformer](https://mlsanity.com/paper/2103.15691/) (2021)
- [Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition](https://mlsanity.com/paper/1801.07455/) (2018)
- [A Closer Look at Spatiotemporal Convolutions for Action Recognition](https://mlsanity.com/paper/1711.11248/) (2017)
- [The "something something" video database for learning and evaluating visual common sense](https://mlsanity.com/paper/1706.04261/) (2017)
- [Video Swin Transformer](https://mlsanity.com/paper/2106.13230/) (2021)
- [Two-Stream Convolutional Networks for Action Recognition in Videos](https://mlsanity.com/paper/1406.2199/) (2014)
- [HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clips](https://mlsanity.com/paper/1906.03327/) (2019)
- [Multiscale Vision Transformers](https://mlsanity.com/paper/2104.11227/) (2021)
- [TSM: Temporal Shift Module for Efficient Video Understanding](https://mlsanity.com/paper/1811.08383/) (2018)
- [X3D: Expanding Architectures for Efficient Video Recognition](https://mlsanity.com/paper/2004.04730/) (2020)
- [Masked Feature Prediction for Self-Supervised Visual Pre-Training](https://mlsanity.com/paper/2112.09133/) (2021)
- [Rethinking Spatiotemporal Feature Learning: Speed-Accuracy Trade-offs in Video Classification](https://mlsanity.com/paper/1712.04851/) (2017)
- [Temporal Convolutional Networks for Action Segmentation and Detection](https://mlsanity.com/paper/1611.05267/) (2016)
- [Long-term Recurrent Convolutional Networks for Visual Recognition and Description](https://mlsanity.com/paper/1411.4389/) (2014)
- [Time-Contrastive Networks: Self-Supervised Learning from Video](https://mlsanity.com/paper/1704.06888/) (2017)
- [AVA: A Video Dataset of Spatio-temporally Localized Atomic Visual Actions](https://mlsanity.com/paper/1705.08421/) (2017)

## Built on by (most cited first, 28 of 28)

- [SoccerNet-Caption: Dense Video Captioning for Soccer Broadcasts Commentaries](https://mlsanity.com/paper/2304.04565/) (2023)
- [VARS: Video Assistant Referee System for Automated Soccer Decision Making from Multiple Views](https://mlsanity.com/paper/2304.04617/) (2023)
- [A Survey on Generative AI and LLM for Video Generation, Understanding, and Streaming](https://mlsanity.com/paper/2404.16038/) (2024)
- [ShuttleSet: A Human-Annotated Stroke-Level Singles Dataset for Badminton Tactical Analysis](https://mlsanity.com/paper/2306.04948/) (2023)
- [A Survey of Deep Learning in Sports Applications: Perception, Comprehension, and Decision](https://mlsanity.com/paper/2307.03353/) (2023)
- [Action Anticipation from SoccerNet Football Video Broadcasts](https://mlsanity.com/paper/2504.12021/) (2025)
- [SoccerChat: Integrating Multimodal Data for Enhanced Soccer Game Understanding](https://mlsanity.com/paper/2505.16630/) (2025)
- [Survey of Action Recognition, Spotting and Spatio-Temporal Localization in Soccer -- Current Trends and Research Perspectives](https://mlsanity.com/paper/2309.12067/) (2023)
- [Deep learning for action spotting in association football videos](https://mlsanity.com/paper/2410.01304/) (2024)
- [P2ANet: A Dataset and Benchmark for Dense Action Detection from Table Tennis Match Broadcasting Videos](https://mlsanity.com/paper/2207.12730/) (2022)
- [Spatio-temporal Prompting Network for Robust Video Feature Extraction](https://mlsanity.com/paper/2402.02574/) (2024)
- [Gate-Shift-Pose: Enhancing Action Recognition in Sports with Skeleton Information](https://mlsanity.com/paper/2503.04470/) (2025)
- [Video4MRI: An Empirical Study on Brain Magnetic Resonance Image Analytics with CNN-based Video Classification Frameworks](https://mlsanity.com/paper/2302.12688/) (2023)
- [OSL-ActionSpotting: A Unified Library for Action Spotting in Sports Videos](https://mlsanity.com/paper/2407.01265/) (2024)
- [SoccerSynth-Detection: A Synthetic Dataset for Soccer Player Detection](https://mlsanity.com/paper/2501.09281/) (2025)
- [VICTOR: Dataset Copyright Auditing in Video Recognition Systems](https://mlsanity.com/paper/2512.14439/) (2025)
- [SportSkills: Physical Skill Learning from Sports Instructional Videos](https://mlsanity.com/paper/2603.25163/) (2026)
- [ViTs for Action Classification in Videos: An Approach to Risky Tackle Detection in American Football Practice Videos](https://mlsanity.com/paper/2604.01318/) (2026)
- [Identifying Ethical Biases in Action Recognition Models](https://mlsanity.com/paper/2604.17971/) (2026)
- [Challenges in Video-Based Infant Action Recognition: A Critical Examination of the State of the Art](https://mlsanity.com/paper/2311.12300/) (2023)
- [A Simple and Effective Temporal Grounding Pipeline for Basketball Broadcast Footage](https://mlsanity.com/paper/2411.00862/) (2024)
- [Future Aspects in Human Action Recognition: Exploring Emerging Techniques and Ethical Influences](https://mlsanity.com/paper/2412.12990/) (2024)
- [HFGCN:Hypergraph Fusion Graph Convolutional Networks for Skeleton-Based Action Recognition](https://mlsanity.com/paper/2501.11007/) (2025)
- [TrajSV: A Trajectory-based Model for Sports Video Representations and Applications](https://mlsanity.com/paper/2508.11569/) (2025)
- [Sparse Transformer for Ultra-sparse Sampled Video Compressive Sensing](https://mlsanity.com/paper/2509.08228/) (2025)
- [Multi-Camera Self-Calibration in Sports Motion Capture: Leveraging Human and Stick Poses](https://mlsanity.com/paper/2604.17567/) (2026)
- [Cross-view Multimodal Vision-Based Assessment Framework for Traditional Chinese Medicine Rehabilitation Training](https://mlsanity.com/paper/2606.28104/) (2026)
- [Throwing4: Phase-Aware Pose Modeling for Throw-Distance Prediction in Elite Athletics](https://mlsanity.com/paper/2610.16383/) (2026)

## Abstract (arXiv)

To understand human behaviors, action recognition based on videos is a common approach. Compared with image-based action recognition, videos provide much more information. Reducing the ambiguity of actions and in the last decade, many works focused on datasets, novel models and learning approaches have improved video action recognition to a higher level. However, there are challenges and unsolved problems, in particular in sports analytics where data collection and labeling are more sophisticated, requiring sport professionals to annotate data. In addition, the actions could be extremely fast and it becomes difficult to recognize them. Moreover, in team sports like football and basketball, one action could involve multiple players, and to correctly recognize them, we need to analyse all players, which is relatively complicated. In this paper, we present a survey on video action recognition for sports analytics. We introduce more than ten types of sports, including team sports, such as football, basketball, volleyball, hockey and individual sports, such as figure skating, gymnastics, table tennis, tennis, diving and badminton. Then we compare numerous existing frameworks for sports analysis to present status quo of video action recognition in both team sports and individual sports. Finally, we discuss the challenges and unsolved problems in this area and to facilitate sports analytics, we develop a toolbox using PaddlePaddle, which supports football, basketball, table tennis and figure skating action recognition.