mlsanityData to 7 Oct 2026

Enabling Factorized Piano Music Modeling and Generation with the MAESTRO Dataset

arXiv 1810.12247 ↗First posted 29 Oct 2018cs.SD cs.LG eess.AScited by 215

Not on the map. Why?

Cited by 25 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Generating musical audio directly with neural networks is notoriously difficult because it requires coherently modeling structure at many different timescales. Fortunately, most music is also highly structured and can be represented as discrete note events played on musical instruments. Herein, we show that by using notes as an intermediate representation, we can train a suite of models capable of transcribing, composing, and synthesizing audio waveforms with coherent musical structure on timescales spanning six orders of magnitude (~0.1 ms to ~100 s), a process we call Wave2Midi2Wave. This large advance in the state of the art is enabled by our release of the new MAESTRO (MIDI and Audio Edited for Synchronous TRacks and Organization) dataset, composed of over 172 hours of virtuosic piano performances captured with fine alignment (~3 ms) between note labels and audio waveforms. The networks and the dataset together present a promising approach toward creating new expressive and interpretable neural models of music.

Key facts. Cited by 215 papers in the map (data through 7 Oct 2026). First posted on arXiv 29 Oct 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 24; 2022: 35; 2023: 29; 2024: 42; 2025: 41; 2026: 44.

215 papers.

Sort:

By area:

  1. 2022AudioLM: a Language Modeling Approach to Audio Generationcited by 611
  2. 2023MusicLM: Generating Music From Textcited by 442
  3. 2021Symbolic Music Generation with Diffusion Modelscited by 152
  4. 2021MusicBERT: Symbolic Music Understanding with Large-Scale Pre-Trainingcited by 89
  5. 2022HEAR: Holistic Evaluation of Audio Representationscited by 77
  6. 2021MT3: Multi-Task Multitrack Music Transcriptioncited by 70
  7. 2022Solving Audio Inverse Problems with a Diffusion Modelcited by 62
  8. 2022General-purpose, long-context autoregressive modeling with Perceiver ARcited by 60
  9. 2023SingSong: Generating musical accompaniments from singingcited by 54
  10. 2024Symbolic Music Generation with Non-Differentiable Rule Guided Diffusioncited by 52
  11. 2021Sequence-to-Sequence Piano Transcription with Transformerscited by 48
  12. 2022Multi-instrument Music Synthesis with Spectrogram Diffusioncited by 42
38 more
  1. 2024VidMuse: A Simple Video-to-Music Generation Framework with Long-Short-Term Modelingcited by 39
  2. 2025Towards Holistic Evaluation of Large Audio-Language Models: A Comprehensive Surveycited by 38
  3. 2024LLMs Meet Multimodal Generation and Editing: A Surveycited by 35
  4. 2024Foundation Models for Music: A Surveycited by 34
  5. 2024MusicMagus: Zero-Shot Text-to-Music Editing via Diffusion Modelscited by 32
  6. 2021MIDI-DDSP: Detailed Control of Musical Performance via Hierarchical Modelingcited by 31
  7. 2022Video Background Music Generation: Dataset, Method and Evaluationcited by 28
  8. 2022A Lightweight Instrument-Agnostic Model for Polyphonic Note Transcription and Multipitch Estimationcited by 27
  9. 2023miditok: A Python package for MIDI file tokenizationcited by 26
  10. 2022Unaligned Supervision For Automatic Music Transcription in The Wildcited by 25
  11. 2022Musika! Fast Infinite Waveform Music Generationcited by 24
  12. 2022Melody transcription via generative pre-trainingcited by 23
  13. 2025Training-Free Guidance Beyond Differentiability: Scalable Path Steering with Tree Search in Diffusion and Flow Modelscited by 23
  14. 2023Automatic Piano Transcription with Hierarchical Frequency-Time Transformercited by 22
  15. 2023Blind Audio Bandwidth Extension: A Diffusion-Based Zero-Shot Approachcited by 21
  16. 2021ReconVAT: A Semi-Supervised Automatic Music Transcription Framework for Low-Resource Real-World Datacited by 20
  17. 2021DadaGP: A Dataset of Tokenized GuitarPro Songs for Sequence Modelscited by 20
  18. 2022Controllable Data Generation by Deep Learning: A Reviewcited by 20
  19. 2023Byte Pair Encoding for Symbolic Musiccited by 19
  20. 2024MidiCaps: A large-scale MIDI dataset with text captionscited by 19
  21. 2021A Generative Model for Raw Audio Using Transformer Architecturescited by 18
  22. 2023V2Meow: Meowing to the Visual Beat via Video-to-Music Generationcited by 18
  23. 2024textttdattri: A Library for Efficient Data Attributioncited by 18
  24. 2024A Data-Driven Analysis of Robust Automatic Piano Transcriptioncited by 15
  25. 2024High Resolution Guitar Transcription via Domain Adaptationcited by 15
  26. 2024PDMX: A Large-Scale Public Domain MusicXML Dataset for Symbolic Music Processingcited by 15
  27. 2024Scoring Time Intervals using Non-Hierarchical Transformer For Automatic Piano Transcriptioncited by 14
  28. 2024MusicFlow: Cascaded Flow Matching for Text Guided Music Generationcited by 14
  29. 2021Omnizart: A General Toolbox for Automatic Music Transcriptioncited by 13
  30. 2022DiffRoll: Diffusion-based Generative Music Transcription with Unsupervised Pretraining Capabilitycited by 13
  31. 2024YourMT3+: Multi-instrument Music Transcription with Enhanced Transformer Architectures and Cross-dataset Stem Augmentationcited by 13
  32. 2021BERT-like Pre-training for Symbolic Piano Music Classification Taskscited by 12
  33. 2022Deep Performer: Score-to-Audio Music Performance Synthesiscited by 12
  34. 2022Symbolic music generation conditioned on continuous-valued emotionscited by 12
  35. 2022Concept-Based Techniques for "Musicologist-friendly" Explanations in a Deep Music Classifiercited by 12
  36. 2023Multitrack Music Transcription with a Time-Frequency Perceivercited by 12
  37. 2024A Diffusion-Based Generative Equalizer for Music Restorationcited by 12
  38. 2024PianoBART: Symbolic Piano Music Generation and Understanding with Large-Scale Pre-Trainingcited by 12

50 most cited of 215 All 215 (JSON)