# Action Quality Assessment using Siamese Network-Based Deep Metric Learning

arXiv: https://arxiv.org/abs/2002.12096 · HTML: https://mlsanity.com/paper/2002.12096/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Feb 2020.

## Built on by (most cited first, 12 of 12)

- [Skating-Mixer: Long-Term Sport Audio-Visual Modeling with MLPs](https://mlsanity.com/paper/2203.03990/) (2022)
- [A Comprehensive Survey of Action Quality Assessment: Method and Benchmark](https://mlsanity.com/paper/2412.11149/) (2024)
- [Piano Skills Assessment](https://mlsanity.com/paper/2101.04884/) (2021)
- [Hierarchical NeuroSymbolic Approach for Comprehensive and Explainable Action Quality Assessment](https://mlsanity.com/paper/2403.13798/) (2024)
- [Continual Action Assessment via Task-Consistent Score-Discriminative Feature Distribution Modeling](https://mlsanity.com/paper/2309.17105/) (2023)
- [Learning Sparse Temporal Video Mapping for Action Quality Assessment in Floor Gymnastics](https://mlsanity.com/paper/2301.06103/) (2023)
- [A Decade of Action Quality Assessment: Largest Systematic Survey of Trends, Challenges, and Future Directions](https://mlsanity.com/paper/2502.02817/) (2025)
- [Modelling the Distribution of Human Motion for Sign Language Assessment](https://mlsanity.com/paper/2408.10073/) (2024)
- [Hand Hygiene Assessment via Joint Step Segmentation and Key Action Scorer](https://mlsanity.com/paper/2209.12221/) (2022)
- [Learning to Score Sign Language with Two-stage Method](https://mlsanity.com/paper/2404.10383/) (2024)
- [Attention-Driven Multimodal Alignment for Long-term Action Quality Assessment](https://mlsanity.com/paper/2507.21945/) (2025)
- [Optimization- and AI-based approaches to academic quality quantification for transparent academic recruitment: part 1-model development](https://mlsanity.com/paper/2305.05460/) (2023)

## Abstract (arXiv)

Automated vision-based score estimation models can be used as an alternate opinion to avoid judgment bias. In the past works the score estimation models were learned by regressing the video representations to the ground truth score provided by the judges. However such regression-based solutions lack interpretability in terms of giving reasons for the awarded score. One solution to make the scores more explicable is to compare the given action video with a reference video. This would capture the temporal variations w.r.t. the reference video and map those variations to the final score. In this work, we propose a new action scoring system as a two-phase system: (1) A Deep Metric Learning Module that learns similarity between any two action videos based on their ground truth scores given by the judges; (2) A Score Estimation Module that uses the first module to find the resemblance of a video to a reference video in order to give the assessment score. The proposed scoring model has been tested for Olympics Diving and Gymnastic vaults and the model outperforms the existing state-of-the-art scoring models.