# ViSTec: Video Modeling for Sports Technique Recognition and Tactical Analysis

arXiv: https://arxiv.org/abs/2402.15952 · HTML: https://mlsanity.com/paper/2402.15952/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 31 references that are in the map.
- First posted on arXiv 25 Feb 2024.

## Builds on (most cited first, 17 of 17)

- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Training](https://mlsanity.com/paper/2203.12602/) (2022)
- [Temporal Convolutional Networks for Action Segmentation and Detection](https://mlsanity.com/paper/1611.05267/) (2016)
- [MS-TCN: Multi-Stage Temporal Convolutional Network for Action Segmentation](https://mlsanity.com/paper/1903.01945/) (2019)
- [BMN: Boundary-Matching Network for Temporal Action Proposal Generation](https://mlsanity.com/paper/1907.09702/) (2019)
- [Temporal Segment Networks for Action Recognition in Videos](https://mlsanity.com/paper/1705.02953/) (2017)
- [BSN: Boundary Sensitive Network for Temporal Action Proposal Generation](https://mlsanity.com/paper/1806.02964/) (2018)
- [G-TAD: Sub-Graph Localization for Temporal Action Detection](https://mlsanity.com/paper/1911.11462/) (2019)
- [FineGym: A Hierarchical Video Dataset for Fine-grained Action Understanding](https://mlsanity.com/paper/2004.06704/) (2020)
- [ASFormer: Transformer for Action Segmentation](https://mlsanity.com/paper/2110.08568/) (2021)
- [CDC: Convolutional-De-Convolutional Networks for Precise Temporal Action Localization in Untrimmed Videos](https://mlsanity.com/paper/1703.01515/) (2017)
- [Fast Learning of Temporal Action Proposal via Dense Boundary Generator](https://mlsanity.com/paper/1911.04127/) (2019)
- [Action Segmentation with Joint Self-Supervised Temporal Domain Adaptation](https://mlsanity.com/paper/2003.02824/) (2020)
- [TrackNet: A Deep Learning Network for Tracking High-speed and Tiny Objects in Sports Applications](https://mlsanity.com/paper/1907.03698/) (2019)
- [Vid2Player: Controllable Video Sprites that Behave and Appear like Professional Tennis Players](https://mlsanity.com/paper/2008.04524/) (2020)
- [Coarse to Fine Multi-Resolution Temporal Convolutional Network](https://mlsanity.com/paper/2105.10859/) (2021)
- [VIRD: Immersive Match Video Analysis for High-Performance Badminton Coaching](https://mlsanity.com/paper/2307.12539/) (2023)

## Built on by (most cited first, 7 of 7)

- [SPORTU: A Comprehensive Sports Understanding Benchmark for Multimodal Large Language Models](https://mlsanity.com/paper/2410.08474/) (2024)
- [F3Set: Towards Analyzing Fast, Frequent, and Fine-grained Events from Videos](https://mlsanity.com/paper/2504.08222/) (2025)
- [CLOT: Closed Loop Optimal Transport for Unsupervised Action Segmentation](https://mlsanity.com/paper/2507.03539/) (2025)
- [2by2: Weakly-Supervised Learning for Global Action Segmentation](https://mlsanity.com/paper/2412.12829/) (2024)
- [Deep Learning for Sports Video Event Detection: Tasks, Datasets, Methods, and Challenges](https://mlsanity.com/paper/2505.03991/) (2025)
- [Extended OpenTT Games Dataset: A table tennis dataset for fine-grained shot type and point outcome](https://mlsanity.com/paper/2512.19327/) (2025)
- [FIS-OT: Feature-Induced Optimal Transport for Unsupervised Action Segmentation](https://mlsanity.com/paper/2608.29980/) (2026)

## Abstract (arXiv)

The immense popularity of racket sports has fueled substantial demand in tactical analysis with broadcast videos. However, existing manual methods require laborious annotation, and recent attempts leveraging video perception models are limited to low-level annotations like ball trajectories, overlooking tactics that necessitate an understanding of stroke techniques. State-of-the-art action segmentation models also struggle with technique recognition due to frequent occlusions and motion-induced blurring in racket sports videos. To address these challenges, We propose ViSTec, a Video-based Sports Technique recognition model inspired by human cognition that synergizes sparse visual data with rich contextual insights. Our approach integrates a graph to explicitly model strategic knowledge in stroke sequences and enhance technique recognition with contextual inductive bias. A two-stage action perception model is jointly trained to align with the contextual knowledge in the graph. Experiments demonstrate that our method outperforms existing models by a significant margin. Case studies with experts from the Chinese national table tennis team validate our model's capacity to automate analysis for technical actions and tactical strategies. More details are available at: https://ViSTec2024.github.io/.