mlsanityData to 14 Oct 2026

Interpreting Graphic Notation with MusicLDM: An AI Improvisation of Cornelius Cardew's Treatise

arXiv 2412.08944 ↗First posted 12 Dec 2024cs.SD cs.LG eess.AScited by 0

Not on the map. Why?

Abstract (arXiv)

This work presents a novel method for composing and improvising music inspired by Cornelius Cardew's Treatise, using AI to bridge graphic notation and musical expression. By leveraging OpenAI's ChatGPT to interpret the abstract visual elements of Treatise, we convert these graphical images into descriptive textual prompts. These prompts are then input into MusicLDM, a pre-trained latent diffusion model designed for music generation. We introduce a technique called "outpainting," which overlaps sections of AI-generated music to create a seamless and cohesive composition. We demostrate a new perspective on performing and interpreting graphic scores, showing how AI can transform visual stimuli into sound and expand the creative possibilities in contemporary/experimental music composition. Musical pieces are available at https://bit.ly/TreatiseAI

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 17 of its 20 references that are in the map. First posted on arXiv 12 Dec 2024.

Builds on

17 of 20 references.

References · 17

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  3. 2020DiffWave: A Versatile Diffusion Model for Audio Synthesiscited by 1,354
  4. 2020Jukebox: A Generative Model for Musiccited by 542
  5. 2023MultiDiffusion: Fusing Diffusion Paths for Controlled Image Generationcited by 536
  6. 2023Simple and Controllable Music Generationcited by 488
  7. 2023MusicLM: Generating Music From Textcited by 448
  8. 2023AudioLDM 2: Learning Holistic Audio Generation with Self-supervised Pretrainingcited by 344
  9. 2018Adversarial Audio Synthesiscited by 225
  10. 2018Enabling Factorized Piano Music Modeling and Generation with the MAESTRO Datasetcited by 217
  11. 2016SampleRNN: An Unconditional End-to-End Neural Audio Generation Modelcited by 110
  12. 2023Mustango: Toward Controllable Text-to-Music Generationcited by 100
5 more
  1. 2023Music ControlNet: Multiple Time-varying Controls for Music Generationcited by 83
  2. 2024DITTO: Diffusion Inference-Time T-Optimization for Music Generationcited by 73
  3. 2022Multi-instrument Music Synthesis with Spectrogram Diffusioncited by 42
  4. 2023Polyffusion: A Diffusion Model for Polyphonic Score Generation with Internal and External Controlscited by 35
  5. 2023Controllable Music Production with Diffusion Models and Guidance Gradientscited by 29

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Towards an AI Musician: Synthesizing Sheet Music Problems for Musical Reasoning
  2. 2025EngravingGNN: A Hybrid Graph Neural Network for End-to-End Piano Score Engraving
  3. 2026Art2Mus: Artwork-to-Music Generation via Visual Conditioning and Large-Scale Cross-Modal Alignment
  4. 2024MeLFusion: Synthesizing Music from Image and Language Cues using Diffusion Models
  5. 2026Generative Modeling of Bach-Style Symbolic Music: A Comparative Study of Autoregressive, Latent-Variable, and Adversarial Approaches
  6. 2024ByteComposer: a Human-like Melody Composition Method based on Language Model Agent
  7. 2023Structured Multi-Track Accompaniment Arrangement via Style Prior Modelling
  8. 2026ONOTE: Hypergraph-Grounded Omnimodal Reasoning for Computational Music Science