# Attention-Driven Multimodal Alignment for Long-term Action Quality Assessment

arXiv: https://arxiv.org/abs/2507.21945 · HTML: https://mlsanity.com/paper/2507.21945/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 44 references that are in the map.
- First posted on arXiv 29 Jul 2025.

## Builds on (most cited first, 28 of 28)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [AST: Audio Spectrogram Transformer](https://mlsanity.com/paper/2104.01778/) (2021)
- [Audio-Visual Scene Analysis with Self-Supervised Multisensory Features](https://mlsanity.com/paper/1804.03641/) (2018)
- [Listen to Look: Action Recognition by Previewing Audio](https://mlsanity.com/paper/1912.04487/) (2019)
- [UMT: Unified Multi-modal Transformers for Joint Video Moment Retrieval and Highlight Detection](https://mlsanity.com/paper/2203.12745/) (2022)
- [Audiovisual SlowFast Networks for Video Recognition](https://mlsanity.com/paper/2001.08740/) (2020)
- [FineDiving: A Fine-grained Dataset for Procedure-aware Action Quality Assessment](https://mlsanity.com/paper/2204.03646/) (2022)
- [What and How Well You Performed? A Multitask Learning Approach to Action Quality Assessment](https://mlsanity.com/paper/1904.04346/) (2019)
- [Learning To Score Olympic Events](https://mlsanity.com/paper/1611.05125/) (2016)
- [Temporal Alignment Networks for Long-term Video](https://mlsanity.com/paper/2204.02968/) (2022)
- [Group-aware Contrastive Regression for Action Quality Assessment](https://mlsanity.com/paper/2108.07797/) (2021)
- [Learning to score the figure skating sports videos](https://mlsanity.com/paper/1802.02774/) (2018)
- [The Pros and Cons: Rank-aware Temporal Attention for Skill Determination in Long Videos](https://mlsanity.com/paper/1812.05538/) (2018)
- [TSA-Net: Tube Self-Attention Network for Action Quality Assessment](https://mlsanity.com/paper/2201.03746/) (2022)
- [Action Quality Assessment with Temporal Parsing Transformer](https://mlsanity.com/paper/2207.09270/) (2022)
- [LOGO: A Long-Form Video Dataset for Group Action Quality Assessment](https://mlsanity.com/paper/2404.05029/) (2024)
- [Video and Accelerometer-Based Motion Analysis for Automated Surgical Skills Assessment](https://mlsanity.com/paper/1702.07772/) (2017)
- [Egocentric Deep Multi-Channel Audio-Visual Active Speaker Localization](https://mlsanity.com/paper/2201.01928/) (2022)
- [MSAF: Multimodal Split Attention Fusion](https://mlsanity.com/paper/2012.07175/) (2020)
- [Skating-Mixer: Long-Term Sport Audio-Visual Modeling with MLPs](https://mlsanity.com/paper/2203.03990/) (2022)
- [Multimodal Action Quality Assessment](https://mlsanity.com/paper/2402.09444/) (2024)
- [Temporal and cross-modal attention for audio-visual zero-shot learning](https://mlsanity.com/paper/2207.09966/) (2022)
- [Action Quality Assessment using Siamese Network-Based Deep Metric Learning](https://mlsanity.com/paper/2002.12096/) (2020)
- [Cross-Modal Global Interaction and Local Alignment for Audio-Visual Speech Recognition](https://mlsanity.com/paper/2305.09212/) (2023)
- [AlignVSR: Audio-Visual Cross-Modal Alignment for Visual Speech Recognition](https://mlsanity.com/paper/2410.16438/) (2024)
- [Cross-Attention is Not Always Needed: Dynamic Cross-Attention for Audio-Visual Dimensional Emotion Recognition](https://mlsanity.com/paper/2403.19554/) (2024)

## Built on by (most cited first, 2 of 2)

- [MLCR: Multi-Level Cue Refinement for Long-Term Multimodal Action Quality Assessment](https://mlsanity.com/paper/2605.08945/) (2026)
- [MotionHalluc: Diagnosing Kinematic Hallucinations in Fine-Grained Motion Reasoning](https://mlsanity.com/paper/2606.23061/) (2026)

## Abstract (arXiv)

Long-term action quality assessment (AQA) focuses on evaluating the quality of human activities in videos lasting up to several minutes. This task plays an important role in the automated evaluation of artistic sports such as rhythmic gymnastics and figure skating, where both accurate motion execution and temporal synchronization with background music are essential for performance assessment. However, existing methods predominantly fall into two categories: unimodal approaches that rely solely on visual features, which are inadequate for modeling multimodal cues like music; and multimodal approaches that typically employ simple feature-level contrastive fusion, overlooking deep cross-modal collaboration and temporal dynamics. As a result, they struggle to capture complex interactions between modalities and fail to accurately track critical performance changes throughout extended sequences. To address these challenges, we propose the Long-term Multimodal Attention Consistency Network (LMAC-Net). LMAC-Net introduces a multimodal attention consistency mechanism to explicitly align multimodal features, enabling stable integration of visual and audio information and enhancing feature representations. Specifically, we introduce a multimodal local query encoder module to capture temporal semantics and cross-modal relations, and use a two-level score evaluation for interpretable results. In addition, attention-based and regression-based losses are applied to jointly optimize multimodal alignment and score fusion. Experiments conducted on the RG and Fis-V datasets demonstrate that LMAC-Net significantly outperforms existing methods, validating the effectiveness of our proposed approach.