mlsanityData to 7 Oct 2026

Evaluating generative audio systems and their metrics

arXiv 2209.00130 ↗First posted 31 Aug 2022cs.SD cs.LG eess.AScited by 16

Not on the map. Why?

Abstract (arXiv)

Recent years have seen considerable advances in audio synthesis with deep generative models. However, the state-of-the-art is very difficult to quantify; different studies often use different evaluation methodologies and different metrics when reporting results, making a direct comparison to other systems difficult if not impossible. Furthermore, the perceptual relevance and meaning of the reported metrics in most cases unknown, prohibiting any conclusive insights with respect to practical usability and audio quality. This paper presents a study that investigates state-of-the-art approaches side-by-side with (i) a set of previously proposed objective metrics for audio reconstruction, and with (ii) a listening study. The results indicate that currently used objective metrics are insufficient to describe the perceptual quality of current systems.

Key facts. Cited by 16 papers in the map (data through 7 Oct 2026). Builds on 17 of its 30 references that are in the map. First posted on arXiv 31 Aug 2022.

Builds on

17 of 30 references.

References · 17

  1. 2016Improved Techniques for Training GANscited by 2,938
  2. 2016WaveNet: A Generative Model for Raw Audiocited by 1,710
  3. 2020DiffWave: A Versatile Diffusion Model for Audio Synthesiscited by 1,339
  4. 2018Demystifying MMD GANscited by 1,268
  5. 2017Tacotron: Towards End-to-End Speech Synthesiscited by 487
  6. 2018WaveGlow: A Flow-based Generative Network for Speech Synthesiscited by 308
  7. 2015A note on the evaluation of generative modelscited by 291
  8. 2017Neural Audio Synthesis of Musical Notes with WaveNet Autoencoderscited by 222
  9. 2018Adversarial Audio Synthesiscited by 222
  10. 2020DDSP: Differentiable Digital Signal Processingcited by 177
  11. 2019GANSynth: Adversarial Neural Audio Synthesiscited by 114
  12. 2018On GANs and GMMscited by 62
5 more
  1. 2021CRASH: Raw Audio Score-based Generative Modeling for Controllable High-resolution Drum Sound Synthesiscited by 37
  2. 2020Comparing Representations for Audio Synthesis Using Generative Adversarial Networkscited by 13
  3. 2021Neural Waveshaping Synthesiscited by 12
  4. 2021DarkGAN: Exploiting Knowledge Distillation for Comprehensible Audio Synthesis with GANscited by 5
  5. 2021VQCPC-GAN: Variable-Length Adversarial Audio Synthesis Using Vector-Quantized Contrastive Predictive Codingcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 5; 2024: 5; 2025: 5; 2026: 1.

16 papers.

Sort:

By area:

  1. 2023Multi-Source Diffusion Models for Simultaneous Music Generation and Separationcited by 63
  2. 2025Aligning Text-to-Music Evaluation with Human Preferencescited by 20
  3. 2025Benchmarking Music Generation Models and Metrics via Human Preference Studiescited by 15
  4. 2023Siamese SIREN: Audio Compression with Implicit Neural Representationscited by 11
  5. 2025Survey on the Evaluation of Generative Models in Musiccited by 10
  6. 2023Multi-modal Latent Diffusioncited by 9
  7. 2024Subtractive Training for Music Stem Insertion using Latent Diffusion Modelscited by 2
  8. 2023What You Hear Is What You See: Audio Quality Metrics From Image Quality Metricscited by 1
  9. 2023Data is Overrated: Perceptual Metrics Can Lead Learning in the Absence of Training Datacited by 1
  10. 2024Latent CLAP Loss for Better Foley Sound Synthesiscited by 1
  11. 2024Prevailing Research Areas for Music AI in the Era of Foundation Modelscited by 1
  12. 2026Benchmarking Single-Factor Physical Video-to-Audio Generationcited by 1
4 more
  1. 2024Source Separation of Multi-source Raw Music using a Residual Quantized Variational Autoencodercited by 0
  2. 2024The Effect of Perceptual Metrics on Music Representation Learning for Genre Classificationcited by 0
  3. 2025MARS: Sound Generation via Multi-Channel Autoregression on Spectrogramscited by 0
  4. 2025Story2MIDI: Emotionally Aligned Music Generation from Textcited by 0