# FLEX: A Largescale Multimodal, Multiview Dataset for Learning Structured Representations for Fitness Action Quality Assessment

arXiv: https://arxiv.org/abs/2506.03198 · HTML: https://mlsanity.com/paper/2506.03198/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 55 references that are in the map.
- First posted on arXiv 2 Jun 2025.

## Builds on (most cited first, 27 of 27)

- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models](https://mlsanity.com/paper/2504.10479/) (2025)
- [LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Models](https://mlsanity.com/paper/2403.13372/) (2024)
- [Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition](https://mlsanity.com/paper/1801.07455/) (2018)
- [MiniCPM-V: A GPT-4V Level MLLM on Your Phone](https://mlsanity.com/paper/2408.01800/) (2024)
- [ActivityNet-QA: A Dataset for Understanding Complex Web Videos via Question Answering](https://mlsanity.com/paper/1906.02467/) (2019)
- [TGIF-QA: Toward Spatio-Temporal Reasoning in Visual Question Answering](https://mlsanity.com/paper/1704.04497/) (2017)
- [Assembly101: A Large-Scale Multi-View Video Dataset for Understanding Procedural Activities](https://mlsanity.com/paper/2203.14712/) (2022)
- [TVQA+: Spatio-Temporal Grounding for Video Question Answering](https://mlsanity.com/paper/1904.11574/) (2019)
- [FineDiving: A Fine-grained Dataset for Procedure-aware Action Quality Assessment](https://mlsanity.com/paper/2204.03646/) (2022)
- [What and How Well You Performed? A Multitask Learning Approach to Action Quality Assessment](https://mlsanity.com/paper/1904.04346/) (2019)
- [Learning To Score Olympic Events](https://mlsanity.com/paper/1611.05125/) (2016)
- [Action Quality Assessment Across Multiple Actions](https://mlsanity.com/paper/1812.06367/) (2018)
- [Group-aware Contrastive Regression for Action Quality Assessment](https://mlsanity.com/paper/2108.07797/) (2021)
- [SkateFormer: Skeletal-Temporal Transformer for Human Action Recognition](https://mlsanity.com/paper/2403.09508/) (2024)
- [TSA-Net: Tube Self-Attention Network for Action Quality Assessment](https://mlsanity.com/paper/2201.03746/) (2022)
- [EgoExo-Fitness: Towards Egocentric and Exocentric Full-Body Action Understanding](https://mlsanity.com/paper/2406.08877/) (2024)
- [Action Quality Assessment with Temporal Parsing Transformer](https://mlsanity.com/paper/2207.09270/) (2022)
- [LOGO: A Long-Form Video Dataset for Group Action Quality Assessment](https://mlsanity.com/paper/2404.05029/) (2024)
- [GAIA: Rethinking Action Quality Assessment for AI-Generated Videos](https://mlsanity.com/paper/2406.06087/) (2024)
- [Domain Knowledge-Informed Self-Supervised Representations for Workout Form Assessment](https://mlsanity.com/paper/2202.14019/) (2022)
- [FLAG3D: A 3D Fitness Activity Dataset with Language Instruction](https://mlsanity.com/paper/2212.04638/) (2022)
- [emg2pose: A Large and Diverse Benchmark for Surface Electromyographic Hand Pose Estimation](https://mlsanity.com/paper/2412.02725/) (2024)
- [Piano Skills Assessment](https://mlsanity.com/paper/2101.04884/) (2021)
- [A Decade of Action Quality Assessment: Largest Systematic Survey of Trends, Challenges, and Future Directions](https://mlsanity.com/paper/2502.02817/) (2025)
- [Towards Activated Muscle Group Estimation in the Wild](https://mlsanity.com/paper/2303.00952/) (2023)

## Built on by (most cited first, 1 of 1)

- [FitAQA: A Benchmark of Fitness Action Quality Assessment for Multimodal Large Language Models](https://mlsanity.com/paper/2608.08736/) (2026)

## Abstract (arXiv)

Action Quality Assessment (AQA) -- the task of quantifying how well an action is performed -- has great potential for detecting errors in gym weight training, where accurate feedback is critical to prevent injuries and maximize gains. Existing AQA datasets, however, are limited to single-view competitive sports and RGB video, lacking multimodal signals and professional assessment of fitness actions. We introduce FLEX, the first large-scale, multimodal, multiview dataset for fitness AQA that incorporates surface electromyography (sEMG). FLEX contains over 7,500 multiview recordings of 20 weight-loaded exercises performed by 38 subjects of diverse skill levels, with synchronized RGB video, 3D pose, sEMG, and physiological signals. Expert annotations are organized into a Fitness Knowledge Graph (FKG) linking actions, key steps, error types, and feedback, supporting a compositional scoring function for interpretable quality assessment. FLEX enables multimodal fusion, cross-modal prediction -- including the novel VideorightarrowEMG task -- and biomechanically oriented representation learning. Building on the FKG, we further introduce FLEX-VideoQA, a structured question-answering benchmark with hierarchical queries that drive cross-modal reasoning in vision-language models. Baseline experiments demonstrate that multimodal inputs, multiview video, and fine-grained annotations significantly enhance AQA performance. FLEX thus advances AQA toward richer multimodal settings and provides a foundation for AI-powered fitness assessment and coaching. Dataset and code are available at hrefhttps://github.com/HaoYin116/FLEXhttps://github.com/HaoYin116/FLEX. Link to Project hrefhttps://haoyin116.github.io/FLEXDatasetpage.