mlsanityData to 7 Oct 2026

DiffRoll: Diffusion-based Generative Music Transcription with Unsupervised Pretraining Capability

arXiv 2210.05148 ↗First posted 11 Oct 2022cs.SD cs.AI cs.LGcited by 13

On the map: Diffusion models → Audio, speech and music

Abstract (arXiv)

In this paper we propose a novel generative approach, DiffRoll, to tackle automatic music transcription (AMT). Instead of treating AMT as a discriminative task in which the model is trained to convert spectrograms into piano rolls, we think of it as a conditional generative task where we train our model to generate realistic looking piano rolls from pure Gaussian noise conditioned on spectrograms. This new AMT formulation enables DiffRoll to transcribe, generate and even inpaint music. Due to the classifier-free nature, DiffRoll is also able to be trained on unpaired datasets where only piano rolls are available. Our experiments show that DiffRoll outperforms its discriminative counterpart by 19 percentage points (ppt.) and our ablation studies also indicate that it outperforms similar existing methods by 4.8 ppt. Source code and demonstration are available https://sony.github.io/DiffRoll/.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Placed in Audio, speech and music, one of 14 directions in Diffusion models. Builds on 26 of its 38 references that are in the map. First posted on arXiv 11 Oct 2022.

Builds on

26 of 38 references.

Closest ancestors · 3

Same direction, newest first

  1. 2022Multi-instrument Music Synthesis with Spectrogram Diffusion11 Jun
  2. 2021Symbolic Music Generation with Diffusion Models30 Mar
  3. 2020DiffWave: A Versatile Diffusion Model for Audio Synthesis21 Sep

Other references · 23

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2020Denoising Diffusion Implicit Modelscited by 9,655
  3. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  4. 2022Classifier-Free Diffusion Guidancecited by 5,717
  5. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  6. 2022Denoising Diffusion Restoration Modelscited by 951
  7. 2020WaveGrad: Estimating Gradients for Waveform Generationcited by 620
  8. 2022Analog Bits: Generating Discrete Data using Diffusion Models with Self-Conditioningcited by 362
  9. 2018Enabling Factorized Piano Music Modeling and Generation with the MAESTRO Datasetcited by 215
  10. 2017MuseGAN: Multi-track Sequential Generative Adversarial Networks for Symbolic Music Generation and Accompanimentcited by 151
  11. 2019GANSynth: Adversarial Neural Audio Synthesiscited by 114
  12. 2017Onsets and Frames: Dual-Objective Piano Transcriptioncited by 93
11 more
  1. 2021MT3: Multi-Task Multitrack Music Transcriptioncited by 70
  2. 2020High-resolution Piano Transcription with Pedals by Regressing Onset and Offset Timescited by 61
  3. 2022Unaligned Supervision For Automatic Music Transcription in The Wildcited by 25
  4. 2021ReconVAT: A Semi-Supervised Automatic Music Transcription Framework for Low-Resource Real-World Datacited by 20
  5. 2020The Effect of Spectrogram Reconstruction on Automatic Music Transcription: An Alternative Approach to Improve Transcription Accuracycited by 10
  6. 2021Revisiting the Onsets and Frames Model with Additive Attentioncited by 9
  7. 2019Deep Unsupervised Drum Transcriptioncited by 7
  8. 2022HPPNet: Modeling the Harmonic Structure and Pitch Invariance in Piano Transcriptioncited by 7
  9. 2022Jointist: Joint Learning for Multi-instrument Transcription and Its Applicationscited by 6
  10. 2020The impact of Audio input representations on neural network based music transcriptioncited by 4
  11. 2019nnAudio: An on-the-fly GPU Audio to Spectrogram Conversion Toolbox Using 1D Convolution Neural Networkscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 5; 2024: 4; 2025: 2; 2026: 1.

13 papers.

Sort:

By area:

  1. 2023Consistency Trajectory Models: Learning Probability Flow ODE Trajectory of Diffusioncited by 448
  2. 2023Multi-Source Diffusion Models for Simultaneous Music Generation and Separationcited by 63
  3. 2023Video2Music: Suitable Music Generation from Videos using an Affective Multimodal Transformer modelcited by 37
  4. 2023Polyffusion: A Diffusion Model for Polyphonic Score Generation with Internal and External Controlscited by 35
  5. 2024DExter: Learning and Controlling Performance Expression with Diffusion Modelscited by 11
  6. 2024PianoMotion10M: Dataset and Benchmark for Hand Motion Generation in Piano Performancecited by 5
  7. 2022A Survey on Artificial Intelligence for Music Generation: Agents, Domains and Perspectivescited by 3
  8. 2025D3RM: A Discrete Denoising Diffusion Refinement Model for Piano Transcriptioncited by 3
  9. 2023Annotation-free Automatic Music Transcription with Scalable Synthetic Data and Adversarial Domain Confusioncited by 2
  10. 2024MR-MT3: Memory Retaining Multi-Track Music Transcription to Mitigate Instrument Leakagecited by 2
  11. 2024Why Perturbing Symbolic Music is Necessary: Fitting the Distribution of Never-used Notes through a Joint Probabilistic Diffusion Modelcited by 2
  12. 2025Noise-to-Notes: Diffusion-based Generation and Refinement for Automatic Drum Transcriptioncited by 1
1 more
  1. 2026Playability-Aware Audio-to-Tablature Guitar Transcription via Diffusion Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023DiffuseRoll: Multi-track multi-category music generation based on diffusion model
  2. 2023Generating symbolic music using diffusion models
  3. 2024Diff-A-Riff: Musical Accompaniment Co-creation via Latent Diffusion Models
  4. 2025DiffRhythm: Blazingly Fast and Embarrassingly Simple End-to-End Full-Length Song Generation with Latent Diffusion
  5. 2024DITTO: Diffusion Inference-Time T-Optimization for Music Generation
  6. 2025Diff-TONE: Timestep Optimization for iNstrument Editing in Text-to-Music Diffusion Models
  7. 2026D3PIA: A Discrete Denoising Diffusion Model for Piano Accompaniment Generation From Lead sheet
  8. 2023GETMusic: Generating Any Music Tracks with a Unified Representation and Diffusion Framework