# Improving Action Quality Assessment using Weighted Aggregation

arXiv: https://arxiv.org/abs/2102.10555 · HTML: https://mlsanity.com/paper/2102.10555/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 23 references that are in the map.
- First posted on arXiv 21 Feb 2021.

## Builds on (most cited first, 14 of 14)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [The Kinetics Human Action Video Dataset](https://mlsanity.com/paper/1705.06950/) (2017)
- [A Closer Look at Spatiotemporal Convolutions for Action Recognition](https://mlsanity.com/paper/1711.11248/) (2017)
- [Learning Spatio-Temporal Features with 3D Residual Networks for Action Recognition](https://mlsanity.com/paper/1708.07632/) (2017)
- [Large-scale weakly-supervised pre-training for video action recognition](https://mlsanity.com/paper/1905.00561/) (2019)
- [What and How Well You Performed? A Multitask Learning Approach to Action Quality Assessment](https://mlsanity.com/paper/1904.04346/) (2019)
- [Learning To Score Olympic Events](https://mlsanity.com/paper/1611.05125/) (2016)
- [Uncertainty-aware Score Distribution Learning for Action Quality Assessment](https://mlsanity.com/paper/2006.07665/) (2020)
- [Action Quality Assessment Across Multiple Actions](https://mlsanity.com/paper/1812.06367/) (2018)
- [Video-based surgical skill assessment using 3D convolutional neural networks](https://mlsanity.com/paper/1903.02306/) (2019)
- [Spatio-Temporal Channel Correlation Networks for Action Classification](https://mlsanity.com/paper/1806.07754/) (2018)
- [Measuring the Quality of Exercises](https://mlsanity.com/paper/1608.09005/) (2016)

## Built on by (most cited first, 4 of 4)

- [PECoP: Parameter Efficient Continual Pretraining for Action Quality Assessment](https://mlsanity.com/paper/2311.07603/) (2023)
- [Interpretable Long-term Action Quality Assessment](https://mlsanity.com/paper/2408.11687/) (2024)
- [A Decade of Action Quality Assessment: Largest Systematic Survey of Trends, Challenges, and Future Directions](https://mlsanity.com/paper/2502.02817/) (2025)
- [Unlabeled Action Quality Assessment Based on Multi-dimensional Adaptive Constrained Dynamic Time Warping](https://mlsanity.com/paper/2410.14161/) (2024)

## Abstract (arXiv)

Action quality assessment (AQA) aims at automatically judging human action based on a video of the said action and assigning a performance score to it. The majority of works in the existing literature on AQA divide RGB videos into short clips, transform these clips to higher-level representations using Convolutional 3D (C3D) networks, and aggregate them through averaging. These higher-level representations are used to perform AQA. We find that the current clip level feature aggregation technique of averaging is insufficient to capture the relative importance of clip level features. In this work, we propose a learning-based weighted-averaging technique. Using this technique, better performance can be obtained without sacrificing too much computational resources. We call this technique Weight-Decider(WD). We also experiment with ResNets for learning better representations for action quality assessment. We assess the effects of the depth and input clip size of the convolutional neural network on the quality of action score predictions. We achieve a new state-of-the-art Spearman's rank correlation of 0.9315 (an increase of 0.45%) on the MTL-AQA dataset using a 34 layer (2+1)D ResNet with the capability of processing 32 frame clips, with WD aggregation.