# 3D Pose-Based Temporal Action Segmentation for Figure Skating: A Fine-Grained and Jump Procedure-Aware Annotation Approach

arXiv: https://arxiv.org/abs/2408.16638 · HTML: https://mlsanity.com/paper/2408.16638/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 76 references that are in the map.
- First posted on arXiv 29 Aug 2024.

## Builds on (most cited first, 30 of 36)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [Rescaling Egocentric Vision](https://mlsanity.com/paper/2006.13256/) (2020)
- [Monocular 3D Human Pose Estimation In The Wild Using Improved CNN Supervision](https://mlsanity.com/paper/1611.09813/) (2016)
- [AI Choreographer: Music Conditioned 3D Dance Generation with AIST++](https://mlsanity.com/paper/2101.08779/) (2021)
- [MS-TCN: Multi-Stage Temporal Convolutional Network for Action Segmentation](https://mlsanity.com/paper/1903.01945/) (2019)
- [BoT-SORT: Robust Associations Multi-Pedestrian Tracking](https://mlsanity.com/paper/2206.14651/) (2022)
- [Effective Whole-body Pose Estimation with Two-stages Distillation](https://mlsanity.com/paper/2307.15880/) (2023)
- [Whole-Body Human Pose Estimation in the Wild](https://mlsanity.com/paper/2007.11858/) (2020)
- [ASFormer: Transformer for Action Segmentation](https://mlsanity.com/paper/2110.08568/) (2021)
- [SoccerNet: A Scalable Dataset for Action Spotting in Soccer Videos](https://mlsanity.com/paper/1804.04527/) (2018)
- [What and How Well You Performed? A Multitask Learning Approach to Action Quality Assessment](https://mlsanity.com/paper/1904.04346/) (2019)
- [The IKEA ASM Dataset: Understanding People Assembling Furniture through Actions, Objects and Pose](https://mlsanity.com/paper/2007.00394/) (2020)
- [Temporal Action Segmentation: An Analysis of Modern Techniques](https://mlsanity.com/paper/2210.10352/) (2022)
- [My View is the Best View: Procedure Learning from Egocentric Videos](https://mlsanity.com/paper/2207.10883/) (2022)
- [Action Quality Assessment Across Multiple Actions](https://mlsanity.com/paper/1812.06367/) (2018)
- [The MECCANO Dataset: Understanding Human-Object Interactions from Egocentric Videos in an Industrial-like Domain](https://mlsanity.com/paper/2010.05654/) (2020)
- [Learning to score the figure skating sports videos](https://mlsanity.com/paper/1802.02774/) (2018)
- [MotionAGFormer: Enhancing 3D Human Pose Estimation with a Transformer-GCNFormer Network](https://mlsanity.com/paper/2310.16288/) (2023)
- [Spotting Temporally Precise, Fine-Grained Events in Video](https://mlsanity.com/paper/2207.10213/) (2022)
- [VARS: Video Assistant Referee System for Automated Soccer Decision Making from Multiple Views](https://mlsanity.com/paper/2304.04617/) (2023)
- [Video Pose Distillation for Few-Shot, Fine-Grained Sports Action Recognition](https://mlsanity.com/paper/2109.01305/) (2021)
- [SportsPose -- A Dynamic 3D sports pose dataset](https://mlsanity.com/paper/2304.01865/) (2023)
- [Semi-Supervised Training to Improve Player and Ball Detection in Soccer](https://mlsanity.com/paper/2204.06859/) (2022)
- [MonoTrack: Shuttle trajectory reconstruction from monocular badminton video](https://mlsanity.com/paper/2204.01899/) (2022)
- [T-DEED: Temporal-Discriminability Enhancer Encoder-Decoder for Precise Event Spotting in Sports Videos](https://mlsanity.com/paper/2404.05392/) (2024)
- [Fine-grained Activity Recognition in Baseball Videos](https://mlsanity.com/paper/1804.03247/) (2018)
- [Group Activity Detection from Trajectory and Video Data in Soccer](https://mlsanity.com/paper/2004.10299/) (2020)
- [Ball 3D Localization From A Single Calibrated Image](https://mlsanity.com/paper/2204.00003/) (2022)

## Built on by (most cited first, 7 of 7)

- [AthletePose3D: A Benchmark Dataset for 3D Human Pose Estimation and Kinematic Validation in Athletic Movements](https://mlsanity.com/paper/2503.07499/) (2025)
- [AthleticsPose: Authentic Sports Motion Dataset on Athletic Field and Evaluation of Monocular 3D Pose Estimation Ability](https://mlsanity.com/paper/2507.12905/) (2025)
- [KASportsFormer: Kinematic Anatomy Enhanced Transformer for 3D Human Pose Estimation on Short Sports Scene Video](https://mlsanity.com/paper/2507.20763/) (2025)
- [VIFSS: View-Invariant and Figure Skating-Specific Pose Representation Learning for Temporal Action Segmentation](https://mlsanity.com/paper/2508.10281/) (2025)
- [Avatar4D: Synthesizing Domain-Specific 4D Humans for Real-World Pose Estimation](https://mlsanity.com/paper/2512.16199/) (2025)
- [Spectral Scalpel: Amplifying Adjacent Action Discrepancy via Frequency-Selective Filtering for Skeleton-Based Action Segmentation](https://mlsanity.com/paper/2603.24134/) (2026)
- [Wiring diagram extraction and gluing: a case study in classifying figure skating jumps using 3D dataset](https://mlsanity.com/paper/2607.27598/) (2026)

## Abstract (arXiv)

Understanding human actions from videos is essential in many domains, including sports. In figure skating, technical judgments are performed by watching skaters' 3D movements, and its part of the judging procedure can be regarded as a Temporal Action Segmentation (TAS) task. TAS tasks in figure skating that automatically assign temporal semantics to video are actively researched. However, there is a lack of datasets and effective methods for TAS tasks requiring 3D pose data. In this study, we first created the FS-Jump3D dataset of complex and dynamic figure skating jumps using optical markerless motion capture. We also propose a new fine-grained figure skating jump TAS dataset annotation method with which TAS models can learn jump procedures. In the experimental results, we validated the usefulness of 3D pose features as input and the fine-grained dataset for the TAS model in figure skating. FS-Jump3D Dataset is available at https://github.com/ryota-skating/FS-Jump3D.