mlsanityData to 14 Oct 2026

Libretto: Giving LLM Agents a Sense of Musical Structure

arXiv 2606.22708 ↗First posted 21 Jun 2026cs.SD cs.AIcited by 0

On the map: LLM agents and tool use → Multimodal and embodied agents

Abstract (arXiv)

Generative music systems can now produce impressive audio from text prompts, but audio outputs are difficult to inspect, edit, and diagnose as musical structure. We introduce Libretto, an agent-facing framework for symbolic music generation and revision. Libretto uses an LLM-native grammar with explicit onset slots, voices, and bar-level organization, then evaluates each piece in a corpus-calibrated statistical space over rhythm, harmony, melody, texture, form, and variation. The same structural axes support retrieval, diagnosis, copy-risk control, and iterative self-revision. Across gap filling, reference-guided full-piece generation, gradual morphing, and educational music generation, Libretto turns symbolic music from a raw token sequence into a measurable and editable object for language-model agents.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use. Builds on 14 of its 24 references that are in the map. First posted on arXiv 21 Jun 2026.

Builds on

14 of 24 references.

References · 14

  1. 2023Simple and Controllable Music Generationcited by 488
  2. 2017MuseGAN: Multi-track Sequential Generative Adversarial Networks for Symbolic Music Generation and Accompanimentcited by 151
  3. 2025Meta Audiobox Aesthetics: Unified Automatic Quality Assessment for Speech, Music, and Soundcited by 137
  4. 2018A Hierarchical Latent Vector Model for Learning Long-Term Structure in Musiccited by 131
  5. 2016DeepBach: a Steerable Model for Bach Chorales Generationcited by 91
  6. 2024ChatMusician: Understanding and Generating Music Intrinsically with LLMcited by 68
  7. 2022Museformer: Transformer with Fine- and Coarse-Grained Attention for Music Generationcited by 49
  8. 2023Anticipatory Music Transformercited by 36
  9. 2022Symphony Generation with Permutation Invariant Language Modelcited by 35
  10. 2024MuPT: A Generative Symbolic Music Pretrained Transformercited by 33
  11. 2025NotaGen: Advancing Musicality in Symbolic Music Generation with Large Language Model Training Paradigmscited by 30
  12. 2024Generating Symbolic Music from Natural Language Prompts using an LLM-Enhanced Datasetcited by 9
2 more
  1. 2025ABC-Eval: Benchmarking Large Language Models on Symbolic Music Understanding and Instruction Followingcited by 6
  2. 2025MIDI-LLM: Improving Text-to-MIDI Music Generation via Adapting Large Language Modelscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024ComposerX: Multi-Agent Symbolic Music Composition with LLMs
  2. 2024ByteComposer: a Human-like Melody Composition Method based on Language Model Agent
  3. 2026How Far Can Pretrained LLMs Go in Symbolic Music? Controlled Comparisons of Supervised and Preference-based Adaptation
  4. 2026MI-MIDI: Mechanistic Interpretability of Text-to-MIDI Generation Models via Probing, Lenses and Steering
  5. 2026YuE2: Unifying Symbolic and Audio Music Generation at Frontier Quality
  6. 2026Verifier-Guided Twelve-Tone Composition: A Generate-Verify-Repair Harness for Symbolic Music Generation
  7. 2025An Agent-Based Framework for Automated Higher-Voice Harmony Generation
  8. 2024Improving Controllability and Editability for Pretrained Text-to-Music Generation Models ↗