# Piano Skills Assessment

arXiv: https://arxiv.org/abs/2101.04884 · HTML: https://mlsanity.com/paper/2101.04884/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 61 references that are in the map.
- First posted on arXiv 13 Jan 2021.

## Builds on (most cited first, 20 of 20)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [What and How Well You Performed? A Multitask Learning Approach to Action Quality Assessment](https://mlsanity.com/paper/1904.04346/) (2019)
- [Learning To Score Olympic Events](https://mlsanity.com/paper/1611.05125/) (2016)
- [Foley Music: Learning to Generate Music from Videos](https://mlsanity.com/paper/2007.10984/) (2020)
- [Uncertainty-aware Score Distribution Learning for Action Quality Assessment](https://mlsanity.com/paper/2006.07665/) (2020)
- [Action Quality Assessment Across Multiple Actions](https://mlsanity.com/paper/1812.06367/) (2018)
- [Learning to score the figure skating sports videos](https://mlsanity.com/paper/1802.02774/) (2018)
- [The Pros and Cons: Rank-aware Temporal Attention for Skill Determination in Long Videos](https://mlsanity.com/paper/1812.05538/) (2018)
- [Creating A Multi-track Classical Musical Performance Dataset for Multimodal Music Analysis: Challenges, Insights, and Applications](https://mlsanity.com/paper/1612.08727/) (2016)
- [Audeo: Audio Generation for a Silent Performance Video](https://mlsanity.com/paper/2006.14348/) (2020)
- [Manipulation-skill Assessment from Videos with Spatial Attention Network](https://mlsanity.com/paper/1901.02579/) (2019)
- [Measuring the Quality of Exercises](https://mlsanity.com/paper/1608.09005/) (2016)
- [Action Quality Assessment using Siamese Network-Based Deep Metric Learning](https://mlsanity.com/paper/2002.12096/) (2020)
- [VI-Net: View-Invariant Quality of Human Movement Assessment](https://mlsanity.com/paper/2008.04999/) (2020)
- [HalluciNet-ing Spatiotemporal Representations Using a 2D-CNN](https://mlsanity.com/paper/1912.04430/) (2019)
- [Virtual Piano using Computer Vision](https://mlsanity.com/paper/1910.12539/) (2019)
- [Modeling User Behaviors in Machine Operation Tasks for Adaptive Guidance](https://mlsanity.com/paper/2003.03025/) (2020)
- [A Multi-Task Learning Approach for Human Activity Segmentation and Ergonomics Risk Assessment](https://mlsanity.com/paper/2008.03014/) (2020)

## Built on by (most cited first, 16 of 16)

- [Domain Knowledge-Informed Self-Supervised Representations for Workout Form Assessment](https://mlsanity.com/paper/2202.14019/) (2022)
- [A Comprehensive Survey of Action Quality Assessment: Method and Benchmark](https://mlsanity.com/paper/2412.11149/) (2024)
- [PECoP: Parameter Efficient Continual Pretraining for Action Quality Assessment](https://mlsanity.com/paper/2311.07603/) (2023)
- [ExpertAF: Expert Actionable Feedback from Video](https://mlsanity.com/paper/2408.00672/) (2024)
- [Uncertainty-Driven Action Quality Assessment](https://mlsanity.com/paper/2207.14513/) (2022)
- [A Decade of Action Quality Assessment: Largest Systematic Survey of Trends, Challenges, and Future Directions](https://mlsanity.com/paper/2502.02817/) (2025)
- [TechCoach: Towards Technical-Point-Aware Descriptive Action Coaching](https://mlsanity.com/paper/2411.17130/) (2024)
- [Win-Fail Action Recognition](https://mlsanity.com/paper/2102.07355/) (2021)
- [FineCausal: A Causal-Based Framework for Interpretable Fine-Grained Action Quality Assessment](https://mlsanity.com/paper/2503.23911/) (2025)
- [SkillSight: Efficient First-Person Skill Assessment with Gaze](https://mlsanity.com/paper/2511.19629/) (2025)
- [FLEX: A Largescale Multimodal, Multiview Dataset for Learning Structured Representations for Fitness Action Quality Assessment](https://mlsanity.com/paper/2506.03198/) (2025)
- [Siamese Residual Neural Network for Musical Shape Evaluation in Piano Performance Assessment](https://mlsanity.com/paper/2401.02566/) (2024)
- [Two Web Toolkits for Multimodal Piano Performance Dataset Acquisition and Fingering Annotation](https://mlsanity.com/paper/2509.15222/) (2025)
- [ProSkill: Segment-Level Skill Assessment in Procedural Videos](https://mlsanity.com/paper/2601.20661/) (2026)
- [Two-Stage Multi-Modal Fusion with Adaptive Alignment for Action Quality Assessment](https://mlsanity.com/paper/2607.07438/) (2026)
- [MyoMechanix: Biomechanically-Grounded Compositional Skilled Activity Understanding and Coaching](https://mlsanity.com/paper/2608.26094/) (2026)

## Abstract (arXiv)

Can a computer determine a piano player's skill level? Is it preferable to base this assessment on visual analysis of the player's performance or should we trust our ears over our eyes? Since current CNNs have difficulty processing long video videos, how can shorter clips be sampled to best reflect the players skill level? In this work, we collect and release a first-of-its-kind dataset for multimodal skill assessment focusing on assessing piano player's skill level, answer the asked questions, initiate work in automated evaluation of piano playing skills and provide baselines for future work. Dataset is available from: https://github.com/ParitoshParmar/Piano-Skills-Assessment.