# Towards Evaluating Generative Audio: Insights from Neural Audio Codec Embedding Distances

arXiv: https://arxiv.org/abs/2509.18823 · HTML: https://mlsanity.com/paper/2509.18823/

## Key facts

- Cited by 1 papers in the map (data through 19 Oct 2026).
- Placed in Vocoders and speech synthesis, one of 11 directions in Speech and audio language models.
- Builds on 17 of its 28 references that are in the map.
- First posted on arXiv 23 Sep 2025.

## Placed in

- Speech and audio language models / [Vocoders and speech synthesis](https://mlsanity.com/speech/vocoders-and-speech-synthesis/)

## Builds on (most cited first, 17 of 17)

- [SoundStream: An End-to-End Neural Audio Codec](https://mlsanity.com/paper/2107.03312/) (2021)
- [High Fidelity Neural Audio Compression](https://mlsanity.com/paper/2210.13438/) (2022)
- [Large-scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to-Caption Augmentation](https://mlsanity.com/paper/2211.06687/) (2022)
- [High-Fidelity Audio Compression with Improved RVQGAN](https://mlsanity.com/paper/2306.06546/) (2023)
- [Rethinking FID: Towards a Better Evaluation Metric for Image Generation](https://mlsanity.com/paper/2401.09603/) (2023)
- [Fast Timing-Conditioned Latent Audio Diffusion](https://mlsanity.com/paper/2402.04825/) (2024)
- [Adapting Frechet Audio Distance for Generative Music Evaluation](https://mlsanity.com/paper/2311.01616/) (2023)
- [Natural Language Supervision for General-Purpose Audio Representations](https://mlsanity.com/paper/2309.05767/) (2023)
- [SongEval: A Benchmark Dataset for Song Aesthetics Evaluation](https://mlsanity.com/paper/2505.10793/) (2025)
- [KAD: No More FAD! An Effective and Efficient Evaluation Metric for Audio Generation](https://mlsanity.com/paper/2502.15602/) (2025)
- [Foley Sound Synthesis at the DCASE 2023 Challenge](https://mlsanity.com/paper/2304.12521/) (2023)
- [MusicEval: A Generative Music Dataset with Expert Ratings for Automatic Text-to-Music Evaluation](https://mlsanity.com/paper/2501.10811/) (2025)
- [High Quality Audio Coding with MDCTNet](https://mlsanity.com/paper/2212.04583/) (2022)
- [ODAQ: Open Dataset of Audio Quality](https://mlsanity.com/paper/2401.00197/) (2023)
- [Assessing the Alignment of Audio Representations with Timbre Similarity Ratings](https://mlsanity.com/paper/2507.07764/) (2025)
- [Towards Reliable Objective Evaluation Metrics for Generative Singing Voice Separation Models](https://mlsanity.com/paper/2507.11427/) (2025)
- [Musical Source Separation Bake-Off: Comparing Objective Metrics with Human Perception](https://mlsanity.com/paper/2507.06917/) (2025)

## Built on by (most cited first, 1 of 1)

- [Objective Intelligibility Prediction Using Distance Metrics on Speech Foundation Model Representations](https://mlsanity.com/paper/2609.13046/) (2026)

## Abstract (arXiv)

Neural audio codecs (NACs) achieve low-bitrate compression by learning compact audio representations, which can also serve as features for perceptual quality evaluation. We introduce DACe, an enhanced, higher-fidelity version of the Descript Audio Codec (DAC), trained on diverse real and synthetic tonal data with balanced sampling. We systematically compare Fr'echet Audio Distance (FAD) and Maximum Mean Discrepancy (MMD) on MUSHRA tests across speech, music, and mixed content. FAD consistently outperforms MMD, and embeddings from higher-fidelity NACs (such as DACe) show stronger correlations with human judgments. While CLAP LAION Music (CLAP-M) and OpenL3 Mel128 (OpenL3-128M) embeddings achieve higher correlations, NAC embeddings provide a practical zero-shot approach to audio quality assessment, requiring only unencoded audio for training. These results demonstrate the dual utility of NACs for compression and perceptually informed audio evaluation.