# Survey on the Evaluation of Generative Models in Music

arXiv: https://arxiv.org/abs/2506.05104 · HTML: https://mlsanity.com/paper/2506.05104/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 92 of its 269 references that are in the map.
- First posted on arXiv 5 Jun 2025.

## Builds on (most cited first, 30 of 92)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [Improved Techniques for Training GANs](https://mlsanity.com/paper/1606.03498/) (2016)
- [Extracting Training Data from Large Language Models](https://mlsanity.com/paper/2012.07805/) (2020)
- [Datasheets for Datasets](https://mlsanity.com/paper/1803.09010/) (2018)
- [DiffWave: A Versatile Diffusion Model for Audio Synthesis](https://mlsanity.com/paper/2009.09761/) (2020)
- [Demystifying MMD GANs](https://mlsanity.com/paper/1801.01401/) (2018)
- [Energy and Policy Considerations for Deep Learning in NLP](https://mlsanity.com/paper/1906.02243/) (2019)
- [Extracting Training Data from Diffusion Models](https://mlsanity.com/paper/2301.13188/) (2023)
- [Large-scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to-Caption Augmentation](https://mlsanity.com/paper/2211.06687/) (2022)
- [CNN Architectures for Large-Scale Audio Classification](https://mlsanity.com/paper/1609.09430/) (2016)
- [High Fidelity Neural Audio Compression](https://mlsanity.com/paper/2210.13438/) (2022)
- [PANNs: Large-Scale Pretrained Audio Neural Networks for Audio Pattern Recognition](https://mlsanity.com/paper/1912.10211/) (2019)
- [Jukebox: A Generative Model for Music](https://mlsanity.com/paper/2005.00341/) (2020)
- [Rethinking FID: Towards a Better Evaluation Metric for Image Generation](https://mlsanity.com/paper/2401.09603/) (2023)
- [A note on the evaluation of generative models](https://mlsanity.com/paper/1511.01844/) (2015)
- [Towards the Systematic Reporting of the Energy and Carbon Footprints of Machine Learning](https://mlsanity.com/paper/2002.05651/) (2020)
- [DNSMOS: A Non-Intrusive Perceptual Objective Speech Quality metric to evaluate Noise Suppressors](https://mlsanity.com/paper/2010.15258/) (2020)
- [Large image datasets: A pyrrhic win for computer vision?](https://mlsanity.com/paper/2006.16923/) (2020)
- [Neural Audio Synthesis of Musical Notes with WaveNet Autoencoders](https://mlsanity.com/paper/1704.01279/) (2017)
- [Music Transformer](https://mlsanity.com/paper/1809.04281/) (2018)
- [Are Large Pre-Trained Language Models Leaking Your Personal Information?](https://mlsanity.com/paper/2205.12628/) (2022)
- [Stable Audio Open](https://mlsanity.com/paper/2407.14358/) (2024)
- [How Faithful is your Synthetic Data? Sample-level Metrics for Evaluating and Auditing Generative Models](https://mlsanity.com/paper/2102.08921/) (2021)
- [Fast Timing-Conditioned Latent Audio Diffusion](https://mlsanity.com/paper/2402.04825/) (2024)
- [DDSP: Differentiable Digital Signal Processing](https://mlsanity.com/paper/2001.04643/) (2020)
- [MuseGAN: Multi-track Sequential Generative Adversarial Networks for Symbolic Music Generation and Accompaniment](https://mlsanity.com/paper/1709.06298/) (2017)
- [C-RNN-GAN: Continuous recurrent neural networks with adversarial training](https://mlsanity.com/paper/1611.09904/) (2016)
- [A Hierarchical Latent Vector Model for Learning Long-Term Structure in Music](https://mlsanity.com/paper/1803.05428/) (2018)

## Built on by (most cited first, 10 of 10)

- [UALM: Unified Audio Language Model for Understanding, Generation and Reasoning](https://mlsanity.com/paper/2510.12000/) (2025)
- [TuneJury: An Open Metric for Improving Music Generation Preference Alignment](https://mlsanity.com/paper/2606.17006/) (2026)
- [Efficient Long-Sequence Diffusion Modeling for Symbolic Music Generation](https://mlsanity.com/paper/2603.00576/) (2026)
- [Position: Natural Language Should Not Fully Replace Formal Languages](https://mlsanity.com/paper/2607.20432/) (2026)
- [MusGU+: Toward a Musician-Centered Evaluation Framework and Discovery Tool for Generative Music AI](https://mlsanity.com/paper/2608.30940/) (2026)
- [Echoes of Humanity: Exploring the Perceived Humanness of AI Music](https://mlsanity.com/paper/2509.25601/) (2025)
- [When Noise Lowers The Loss: Rethinking Likelihood-Based Evaluation in Music Large Language Models](https://mlsanity.com/paper/2602.02738/) (2026)
- [RPPNet: Perceptually-Grouped Rhythm-Pitch Primitives for Long-Term Structure Melody Generation via Boundary-Aware Modeling](https://mlsanity.com/paper/2607.19776/) (2026)
- [TTM-Bench: A Framework for Text-to-Music System Performance Benchmarking](https://mlsanity.com/paper/2609.18585/) (2026)
- [Don't CLAP: Are Music-Text Models Bag-of-Words?](https://mlsanity.com/paper/2609.30540/) (2026)

## Abstract (arXiv)

Research on generative systems in music has seen considerable attention and growth in recent years. A variety of attempts have been made to systematically evaluate such systems. We present an interdisciplinary review of the common evaluation targets, methodologies, and metrics for the evaluation of both system output and model use, covering subjective and objective approaches, qualitative and quantitative approaches, as well as empirical and computational methods. We examine the benefits and limitations of these approaches from a musicological, an engineering, and an HCI perspective.