# Action Quality Assessment with Temporal Parsing Transformer

arXiv: https://arxiv.org/abs/2207.09270 · HTML: https://mlsanity.com/paper/2207.09270/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 35 references that are in the map.
- First posted on arXiv 19 Jul 2022.

## Builds on (most cited first, 22 of 22)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition](https://mlsanity.com/paper/1801.07455/) (2018)
- [Temporal Segment Networks: Towards Good Practices for Deep Action Recognition](https://mlsanity.com/paper/1608.00859/) (2016)
- [Temporal Convolutional Networks for Action Segmentation and Detection](https://mlsanity.com/paper/1611.05267/) (2016)
- [Conditional DETR for Fast Training Convergence](https://mlsanity.com/paper/2108.06152/) (2021)
- [Associative Embedding: End-to-End Learning for Joint Detection and Grouping](https://mlsanity.com/paper/1611.05424/) (2016)
- [ASFormer: Transformer for Action Segmentation](https://mlsanity.com/paper/2110.08568/) (2021)
- [What and How Well You Performed? A Multitask Learning Approach to Action Quality Assessment](https://mlsanity.com/paper/1904.04346/) (2019)
- [Learning To Score Olympic Events](https://mlsanity.com/paper/1611.05125/) (2016)
- [Uncertainty-aware Score Distribution Learning for Action Quality Assessment](https://mlsanity.com/paper/2006.07665/) (2020)
- [Action Quality Assessment Across Multiple Actions](https://mlsanity.com/paper/1812.06367/) (2018)
- [Group-aware Contrastive Regression for Action Quality Assessment](https://mlsanity.com/paper/2108.07797/) (2021)
- [Learning to score the figure skating sports videos](https://mlsanity.com/paper/1802.02774/) (2018)
- [The Pros and Cons: Rank-aware Temporal Attention for Skill Determination in Long Videos](https://mlsanity.com/paper/1812.05538/) (2018)
- [Temporal Query Networks for Fine-grained Video Understanding](https://mlsanity.com/paper/2104.09496/) (2021)
- [Weakly Supervised Energy-Based Learning for Action Segmentation](https://mlsanity.com/paper/1909.13155/) (2019)
- [Intra- and Inter-Action Understanding via Temporal Action Parsing](https://mlsanity.com/paper/2005.10229/) (2020)
- [Am I a Baller? Basketball Performance Assessment from First-Person Videos](https://mlsanity.com/paper/1611.05365/) (2016)
- [TSA-Net: Tube Self-Attention Network for Action Quality Assessment](https://mlsanity.com/paper/2201.03746/) (2022)
- [Joint Discovery of Object States and Manipulation Actions](https://mlsanity.com/paper/1702.02738/) (2017)

## Built on by (most cited first, 30 of 31)

- [Vision Transformers for Action Recognition: A Survey](https://mlsanity.com/paper/2209.05700/) (2022)
- [Action Quality Assessment via Hierarchical Pose-guided Multi-stage Contrastive Regression](https://mlsanity.com/paper/2501.03674/) (2025)
- [Multimodal Action Quality Assessment](https://mlsanity.com/paper/2402.09444/) (2024)
- [A Comprehensive Survey of Action Quality Assessment: Method and Benchmark](https://mlsanity.com/paper/2412.11149/) (2024)
- [PECoP: Parameter Efficient Continual Pretraining for Action Quality Assessment](https://mlsanity.com/paper/2311.07603/) (2023)
- [Narrative Action Evaluation with Prompt-Guided Multimodal Interaction](https://mlsanity.com/paper/2404.14471/) (2024)
- [RICA2: Rubric-Informed, Calibrated Assessment of Actions](https://mlsanity.com/paper/2408.02138/) (2024)
- [Continual Action Assessment via Task-Consistent Score-Discriminative Feature Distribution Modeling](https://mlsanity.com/paper/2309.17105/) (2023)
- [PHI: Bridging Domain Shift in Long-Term Action Quality Assessment via Progressive Hierarchical Instruction](https://mlsanity.com/paper/2505.19972/) (2025)
- [Semi-Supervised Teacher-Reference-Student Architecture for Action Quality Assessment](https://mlsanity.com/paper/2407.19675/) (2024)
- [Uncertainty-Driven Action Quality Assessment](https://mlsanity.com/paper/2207.14513/) (2022)
- [Interpretable Long-term Action Quality Assessment](https://mlsanity.com/paper/2408.11687/) (2024)
- [A Decade of Action Quality Assessment: Largest Systematic Survey of Trends, Challenges, and Future Directions](https://mlsanity.com/paper/2502.02817/) (2025)
- [TechCoach: Towards Technical-Point-Aware Descriptive Action Coaching](https://mlsanity.com/paper/2411.17130/) (2024)
- [HuMoCon: Concept Discovery for Human Motion Understanding](https://mlsanity.com/paper/2505.20920/) (2025)
- [QAFE-Net: Quality Assessment of Facial Expressions with Landmark Heatmaps](https://mlsanity.com/paper/2312.00856/) (2023)
- [Learning to Visually Connect Actions and their Effects](https://mlsanity.com/paper/2401.10805/) (2024)
- [Attention-Driven Multimodal Alignment for Long-term Action Quality Assessment](https://mlsanity.com/paper/2507.21945/) (2025)
- [Can Vision Language Models Judge Action Quality? An Empirical Evaluation](https://mlsanity.com/paper/2604.08294/) (2026)
- [ZEAL: Surgical Skill Assessment with Zero-shot Tool Inference Using Unified Foundation Model](https://mlsanity.com/paper/2407.02738/) (2024)
- [Trajectory-guided Motion Perception for Facial Expression Quality Assessment in Neurological Disorders](https://mlsanity.com/paper/2504.09530/) (2025)
- [FLEX: A Largescale Multimodal, Multiview Dataset for Learning Structured Representations for Fitness Action Quality Assessment](https://mlsanity.com/paper/2506.03198/) (2025)
- [CoRe-DA: Contrastive Regression for Unsupervised Domain Adaptation in Surgical Skill Assessment](https://mlsanity.com/paper/2603.29666/) (2026)
- [A Topology-Aware Graph Convolutional Network for Human Pose Similarity and Action Quality Assessment](https://mlsanity.com/paper/2511.01194/) (2025)
- [FineSkiing: A Fine-grained Benchmark for Skiing Action Quality Assessment](https://mlsanity.com/paper/2511.10250/) (2025)
- [ProSkill: Segment-Level Skill Assessment in Procedural Videos](https://mlsanity.com/paper/2601.20661/) (2026)
- [SurgFusion-Net: Diversified Adaptive Multimodal Fusion Network for Surgical Skill Assessment](https://mlsanity.com/paper/2603.00108/) (2026)
- [MLCR: Multi-Level Cue Refinement for Long-Term Multimodal Action Quality Assessment](https://mlsanity.com/paper/2605.08945/) (2026)
- [Latent Visual Diffusion Reasoning with Monte Carlo Tree Search](https://mlsanity.com/paper/2606.27988/) (2026)
- [MoAKE: Toward Unified All-in-One Action Quality Assessment via Mixture of Action Knowledge Experts](https://mlsanity.com/paper/2607.19826/) (2026)

## Abstract (arXiv)

Action Quality Assessment(AQA) is important for action understanding and resolving the task poses unique challenges due to subtle visual differences. Existing state-of-the-art methods typically rely on the holistic video representations for score regression or ranking, which limits the generalization to capture fine-grained intra-class variation. To overcome the above limitation, we propose a temporal parsing transformer to decompose the holistic feature into temporal part-level representations. Specifically, we utilize a set of learnable queries to represent the atomic temporal patterns for a specific action. Our decoding process converts the frame representations to a fixed number of temporally ordered part representations. To obtain the quality score, we adopt the state-of-the-art contrastive regression based on the part representations. Since existing AQA datasets do not provide temporal part-level labels or partitions, we propose two novel loss functions on the cross attention responses of the decoder: a ranking loss to ensure the learnable queries to satisfy the temporal order in cross attention and a sparsity loss to encourage the part representations to be more discriminative. Extensive experiments show that our proposed method outperforms prior work on three public AQA benchmarks by a considerable margin.