mlsanityData to 7 Oct 2026

The Role of Large Language Models in Musicology: Are We Ready to Trust the Machines?

arXiv 2409.01864 ↗First posted 3 Sep 2024cs.SD cs.AI cs.CLcited by 3

On the map: Retrieval-augmented generation → RAG pipeline design and evaluation

Abstract (arXiv)

In this work, we explore the use and reliability of Large Language Models (LLMs) in musicology. From a discussion with experts and students, we assess the current acceptance and concerns regarding this, nowadays ubiquitous, technology. We aim to go one step further, proposing a semi-automatic method to create an initial benchmark using retrieval-augmented generation models and multiple-choice question generation, validated by human experts. Our evaluation on 400 human-validated questions shows that current vanilla LLMs are less reliable than retrieval augmented generation from music dictionaries. This paper suggests that the potential of LLMs in musicology requires musicology driven research that can specialized LLMs by including accurate and reliable domain knowledge.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in RAG pipeline design and evaluation, one of 15 directions in Retrieval-augmented generation. Builds on 10 of its 30 references that are in the map. First posted on arXiv 3 Sep 2024.

Builds on

10 of 30 references.

References · 10

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  3. 2023Qwen Technical Reportcited by 3,273
  4. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,346
  5. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,883
  6. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,800
  7. 2021Ethical and social risks of harm from Language Modelscited by 957
  8. 2023Bias and Fairness in Large Language Models: A Surveycited by 626
  9. 2024Adaptive-RAG: Learning to Adapt Retrieval-Augmented Large Language Models through Question Complexitycited by 437
  10. 2021Measuring Attribution in Natural Language Generation Modelscited by 253

Built on by

Cited by year

Citing papers per year: 2025: 3.

3 papers.

  1. 2025Explain Less, Understand More: Data-Efficient Personalization of Reader-Dependent Jargonscited by 0
  2. 2025MUST-RAG: MUSical Text Question Answering with Retrieval Augmented Generationcited by 0
  3. 2025ArtistMus: A Globally Diverse, Artist-Centric Benchmark for Retrieval-Augmented Music Question Answeringcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024MuChoMusic: Evaluating Music Understanding in Multimodal Audio-Language Models
  2. 2024Can LLMs "Reason" in Music? An Evaluation of LLMs' Capability of Music Understanding and Generation
  3. 2026BASS: Benchmarking Audio LMs for Musical Structure and Semantic Reasoning
  4. 2026How Far Can Pretrained LLMs Go in Symbolic Music? Controlled Comparisons of Supervised and Preference-based Adaptation
  5. 2025Towards Reliable Large Audio Language Model
  6. 2025Music Flamingo: Scaling Music Understanding in Audio Language Models
  7. 2025WildScore: Benchmarking MLLMs in-the-Wild Symbolic Music Reasoning
  8. 2025Exploring GPT's Ability as a Judge in Music Understanding ↗