mlsanityData to 7 Oct 2026

Teaching LLMs Music Theory with In-Context Learning and Chain-of-Thought Prompting: Pedagogical Strategies for Machines

arXiv 2503.22853 ↗First posted 28 Mar 2025cs.SD cs.AIcited by 2

On the map: LLM reasoning → Audio-language model reasoning

Abstract (arXiv)

This study evaluates the baseline capabilities of Large Language Models (LLMs) like ChatGPT, Claude, and Gemini to learn concepts in music theory through in-context learning and chain-of-thought prompting. Using carefully designed prompts (in-context learning) and step-by-step worked examples (chain-of-thought prompting), we explore how LLMs can be taught increasingly complex material and how pedagogical strategies for human learners translate to educating machines. Performance is evaluated using questions from an official Canadian Royal Conservatory of Music (RCM) Level 6 examination, which covers a comprehensive range of topics, including interval and chord identification, key detection, cadence classification, and metrical analysis. Additionally, we evaluate the suitability of various music encoding formats for these tasks (ABC, Humdrum, MEI, MusicXML). All experiments were run both with and without contextual prompts. Results indicate that without context, ChatGPT with MEI performs the best at 52%, while with context, Claude with MEI performs the best at 75%. Future work will further refine prompts and expand to cover more advanced music theory concepts. This research contributes to the broader understanding of teaching LLMs and has applications for educators, students, and developers of AI music tools alike.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Audio-language model reasoning, one of 17 directions in LLM reasoning. Builds on 8 of its 31 references that are in the map. First posted on arXiv 28 Mar 2025.

Builds on

8 of 31 references.

References · 8

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  3. 2022Automatic Chain of Thought Prompting in Large Language Modelscited by 684
  4. 2022Optimizing Prompts for Text-to-Image Generationcited by 204
  5. 2023How Novices Use LLM-Based Code Generators to Solve CS1 Coding Tasks in a Self-Paced Learning Environmentcited by 55
  6. 2024ComposerX: Multi-Agent Symbolic Music Composition with LLMscited by 40
  7. 2024Sheet Music Transformer: End-To-End Optical Music Recognition Beyond Monophonic Transcriptioncited by 20
  8. 2024LLM In-Context Recall is Prompt Dependentcited by 15

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 1.

2 papers.

  1. 2025CSyMR: Benchmarking Compositional Music Information Retrieval in Symbolic Music Reasoningcited by 2
  2. 2026Reasoning4Sciences: Bridging Reasoning Language Models to All Scientific Branchescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Can LLMs "Reason" in Music? An Evaluation of LLMs' Capability of Music Understanding and Generation
  2. 2026How Far Can Pretrained LLMs Go in Symbolic Music? Controlled Comparisons of Supervised and Preference-based Adaptation
  3. 2025ABC-Eval: Benchmarking Large Language Models on Symbolic Music Understanding and Instruction Following
  4. 2024The Role of Large Language Models in Musicology: Are We Ready to Trust the Machines?
  5. 2026Exploring LLMs for South Asian Music Understanding and Generation
  6. 2024Harmonic Reasoning in Large Language Models
  7. 2024Foundation Models for Music: A Survey
  8. 2026MIDI-LLaMA: An Instruction-Following Multimodal LLM for Symbolic Music Understanding