# LLaQo: Towards a Query-Based Coach in Expressive Music Performance Assessment

arXiv: https://arxiv.org/abs/2409.08795 · HTML: https://mlsanity.com/paper/2409.08795/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Sep 2024.

## Built on by (most cited first, 5 of 5)

- [Prevailing Research Areas for Music AI in the Era of Foundation Models](https://mlsanity.com/paper/2409.09378/) (2024)
- [EmoHeal: An End-to-End System for Personalized Therapeutic Music Retrieval from Fine-grained Emotions](https://mlsanity.com/paper/2509.15986/) (2025)
- [Joint Estimation of Piano Dynamics and Metrical Structure with a Multi-task Multi-Scale Network](https://mlsanity.com/paper/2510.18190/) (2025)
- [Profy: Interpretable Visualization of Expertise-Dependent Motor Skills Toward Supporting Piano Practice](https://mlsanity.com/paper/2606.10627/) (2026)
- [Integrating Contextual Embeddings into Evaluation of Expressive MIDI Piano Performances](https://mlsanity.com/paper/2607.27909/) (2026)

## Abstract (arXiv)

Research in music understanding has extensively explored composition-level attributes such as key, genre, and instrumentation through advanced representations, leading to cross-modal applications using large language models. However, aspects of musical performance such as stylistic expression and technique remain underexplored, along with the potential of using large language models to enhance educational outcomes with customized feedback. To bridge this gap, we introduce LLaQo, a Large Language Query-based music coach that leverages audio language modeling to provide detailed and formative assessments of music performances. We also introduce instruction-tuned query-response datasets that cover a variety of performance dimensions from pitch accuracy to articulation, as well as contextual performance understanding (such as difficulty and performance techniques). Utilizing AudioMAE encoder and Vicuna-7b LLM backend, our model achieved state-of-the-art (SOTA) results in predicting teachers' performance ratings, as well as in identifying piece difficulty and playing techniques. Textual responses from LLaQo was moreover rated significantly higher compared to other baseline models in a user study using audio-text matching. Our proposed model can thus provide informative answers to open-ended questions related to musical performance from audio data.