mlsanityData to 14 Oct 2026

Survey on the Evaluation of Generative Models in Music

arXiv 2506.05104 ↗First posted 5 Jun 2025cs.SD cs.AI cs.LGcited by 10

Not on the map. Why?

Abstract (arXiv)

Research on generative systems in music has seen considerable attention and growth in recent years. A variety of attempts have been made to systematically evaluate such systems. We present an interdisciplinary review of the common evaluation targets, methodologies, and metrics for the evaluation of both system output and model use, covering subjective and objective approaches, qualitative and quantitative approaches, as well as empirical and computational methods. We examine the benefits and limitations of these approaches from a musicological, an engineering, and an HCI perspective.

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Builds on 92 of its 269 references that are in the map. First posted on arXiv 5 Jun 2025.

Builds on

92 of 269 references.

References · 92

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  3. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  4. 2016Improved Techniques for Training GANscited by 2,970
  5. 2020Extracting Training Data from Large Language Modelscited by 2,189
  6. 2018Datasheets for Datasetscited by 1,386
  7. 2020DiffWave: A Versatile Diffusion Model for Audio Synthesiscited by 1,354
  8. 2018Demystifying MMD GANscited by 1,293
  9. 2019Energy and Policy Considerations for Deep Learning in NLPcited by 1,254
  10. 2023Extracting Training Data from Diffusion Modelscited by 732
  11. 2022Large-scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to-Caption Augmentationcited by 719
  12. 2016CNN Architectures for Large-Scale Audio Classificationcited by 704
38 more
  1. 2022High Fidelity Neural Audio Compressioncited by 687
  2. 2019PANNs: Large-Scale Pretrained Audio Neural Networks for Audio Pattern Recognitioncited by 574
  3. 2020Jukebox: A Generative Model for Musiccited by 542
  4. 2023Rethinking FID: Towards a Better Evaluation Metric for Image Generationcited by 310
  5. 2015A note on the evaluation of generative modelscited by 292
  6. 2020Towards the Systematic Reporting of the Energy and Carbon Footprints of Machine Learningcited by 249
  7. 2020DNSMOS: A Non-Intrusive Perceptual Objective Speech Quality metric to evaluate Noise Suppressorscited by 241
  8. 2020Large image datasets: A pyrrhic win for computer vision?cited by 237
  9. 2017Neural Audio Synthesis of Musical Notes with WaveNet Autoencoderscited by 225
  10. 2018Music Transformercited by 211
  11. 2022Are Large Pre-Trained Language Models Leaking Your Personal Information?cited by 200
  12. 2024Stable Audio Opencited by 197
  13. 2021How Faithful is your Synthetic Data? Sample-level Metrics for Evaluating and Auditing Generative Modelscited by 196
  14. 2024Fast Timing-Conditioned Latent Audio Diffusioncited by 181
  15. 2020DDSP: Differentiable Digital Signal Processingcited by 178
  16. 2017MuseGAN: Multi-track Sequential Generative Adversarial Networks for Symbolic Music Generation and Accompanimentcited by 151
  17. 2016C-RNN-GAN: Continuous recurrent neural networks with adversarial trainingcited by 132
  18. 2018A Hierarchical Latent Vector Model for Learning Long-Term Structure in Musiccited by 131
  19. 2021Pros and Cons of GAN Evaluation Measures: New Developmentscited by 123
  20. 2019GANSynth: Adversarial Neural Audio Synthesiscited by 116
  21. 2019Lessons from Archives: Strategies for Collecting Sociocultural Data in Machine Learningcited by 116
  22. 2023Art and the science of generative AI: A deeper divecited by 95
  23. 2023MusicLDM: Enhancing Novelty in Text-to-Music Generation Using Beat-Synchronous Mixup Strategiescited by 94
  24. 2023Adapting Frechet Audio Distance for Generative Music Evaluationcited by 92
  25. 2016DeepBach: a Steerable Model for Bach Chorales Generationcited by 91
  26. 2017MidiNet: A Convolutional Generative Adversarial Network for Symbolic-domain Music Generationcited by 72
  27. 2018This Time with Feeling: Learning Expressive Musical Performancecited by 68
  28. 2024ChatMusician: Understanding and Generating Music Intrinsically with LLMcited by 68
  29. 2018The challenge of realistic music generation: modelling raw audio at scalecited by 63
  30. 2018On GANs and GMMscited by 62
  31. 2021On Memorization in Probabilistic Deep Generative Modelscited by 60
  32. 2019LakhNES: Improving multi-instrumental music generation with cross-domain pre-trainingcited by 54
  33. 2020The Jazz Transformer on the Front Line: Exploring the Shortcomings of AI-composed Music through Quantitative Measurescited by 53
  34. 2024OmniFlow: Any-to-Any Generation with Multi-Modal Rectified Flowscited by 46
  35. 2025DiffRhythm: Blazingly Fast and Embarrassingly Simple End-to-End Full-Length Song Generation with Latent Diffusioncited by 46
  36. 2020Music FaderNets: Controllable Music Generation Based On High-Level Features via Low-Level Feature Modellingcited by 45
  37. 2018A Functional Taxonomy of Music Generation Systemscited by 31
  38. 2020Towards GAN Benchmarks Which Require Generalizationcited by 31

50 most cited of 92 All 92 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 8.

10 papers.

  1. 2025UALM: Unified Audio Language Model for Understanding, Generation and Reasoningcited by 10
  2. 2026TuneJury: An Open Metric for Improving Music Generation Preference Alignmentcited by 4
  3. 2026Efficient Long-Sequence Diffusion Modeling for Symbolic Music Generationcited by 1
  4. 2026Position: Natural Language Should Not Fully Replace Formal Languagescited by 1
  5. 2026MusGU+: Toward a Musician-Centered Evaluation Framework and Discovery Tool for Generative Music AIcited by 1
  6. 2025Echoes of Humanity: Exploring the Perceived Humanness of AI Musiccited by 0
  7. 2026When Noise Lowers The Loss: Rethinking Likelihood-Based Evaluation in Music Large Language Modelscited by 0
  8. 2026RPPNet: Perceptually-Grouped Rhythm-Pitch Primitives for Long-Term Structure Melody Generation via Boundary-Aware Modelingcited by 0
  9. 2026TTM-Bench: A Framework for Text-to-Music System Performance Benchmarkingcited by 0
  10. 2026Don't CLAP: Are Music-Text Models Bag-of-Words?cited by 0