# Three-Stream 3D/1D CNN for Fine-Grained Action Classification and Segmentation in Table Tennis

arXiv: https://arxiv.org/abs/2109.14306 · HTML: https://mlsanity.com/paper/2109.14306/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 38 references that are in the map.
- First posted on arXiv 29 Sep 2021.

## Builds on (most cited first, 14 of 14)

- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [Wide Residual Networks](https://mlsanity.com/paper/1605.07146/) (2016)
- [FineGym: A Hierarchical Video Dataset for Fine-grained Action Understanding](https://mlsanity.com/paper/2004.06704/) (2020)
- [PersonLab: Person Pose Estimation and Instance Segmentation with a Bottom-Up, Part-Based, Geometric Embedding Model](https://mlsanity.com/paper/1803.08225/) (2018)
- [2D/3D Pose Estimation and Action Recognition using Multitask Deep Learning](https://mlsanity.com/paper/1802.09232/) (2018)
- [A Short Note on the Kinetics-700-2020 Human Action Dataset](https://mlsanity.com/paper/2010.10864/) (2020)
- [LCR-Net++: Multi-person 2D and 3D Pose Detection in Natural Images](https://mlsanity.com/paper/1803.00455/) (2018)
- [The AVA-Kinetics Localized Human Actions Video Dataset](https://mlsanity.com/paper/2005.00214/) (2020)
- [SharpNet: Fast and Accurate Recovery of Occluding Contours in Monocular Depth Estimation](https://mlsanity.com/paper/1905.08598/) (2019)
- [TTNet: Real-time temporal and spatial video analysis of table tennis](https://mlsanity.com/paper/2004.09927/) (2020)
- [Towards Understanding the Adversarial Vulnerability of Skeleton-based Action Recognition](https://mlsanity.com/paper/2005.07151/) (2020)
- [Activity-conditioned continuous human pose estimation for performance analysis of athletes using the example of swimming](https://mlsanity.com/paper/1802.00634/) (2018)
- [3D attention mechanism for fine-grained classification of table tennis strokes using a Twin Spatio-Temporal Convolutional Neural Networks](https://mlsanity.com/paper/2012.05342/) (2020)

## Built on by (most cited first, 5 of 5)

- [A Survey on Video Action Recognition in Sports: Datasets, Methods and Applications](https://mlsanity.com/paper/2206.01038/) (2022)
- [P2ANet: A Dataset and Benchmark for Dense Action Detection from Table Tennis Match Broadcasting Videos](https://mlsanity.com/paper/2207.12730/) (2022)
- [Table Tennis Stroke Detection and Recognition Using Ball Trajectory Data](https://mlsanity.com/paper/2302.09657/) (2023)
- [Spatio-Temporal CNN baseline method for the Sports Video Task of MediaEval 2021 benchmark](https://mlsanity.com/paper/2112.12074/) (2021)
- [TransNet: A Transfer Learning-Based Network for Human Action Recognition](https://mlsanity.com/paper/2309.06951/) (2023)

## Abstract (arXiv)

This paper proposes a fusion method of modalities extracted from video through a three-stream network with spatio-temporal and temporal convolutions for fine-grained action classification in sport. It is applied to TTStroke-21 dataset which consists of untrimmed videos of table tennis games. The goal is to detect and classify table tennis strokes in the videos, the first step of a bigger scheme aiming at giving feedback to the players for improving their performance. The three modalities are raw RGB data, the computed optical flow and the estimated pose of the player. The network consists of three branches with attention blocks. Features are fused at the latest stage of the network using bilinear layers. Compared to previous approaches, the use of three modalities allows faster convergence and better performances on both tasks: classification of strokes with known temporal boundaries and joint segmentation and classification. The pose is also further investigated in order to offer richer feedback to the athletes.