# Analysis of Speaker Verification Performance Trade-offs with Neural Audio Codec Transmission

arXiv: https://arxiv.org/abs/2509.02771 · HTML: https://mlsanity.com/paper/2509.02771/

## Key facts

- No citing papers in the map yet (data through 19 Oct 2026).
- Placed in Vocoders and speech synthesis, one of 11 directions in Speech and audio language models.
- Builds on 20 of its 26 references that are in the map.
- First posted on arXiv 2 Sep 2025.

## Placed in

- Speech and audio language models / [Vocoders and speech synthesis](https://mlsanity.com/speech/vocoders-and-speech-synthesis/)

## Builds on (most cited first, 20 of 20)

- [Taming Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2012.09841/) (2020)
- [SoundStream: An End-to-End Neural Audio Codec](https://mlsanity.com/paper/2107.03312/) (2021)
- [VoxCeleb: a large-scale speaker identification dataset](https://mlsanity.com/paper/1706.08612/) (2017)
- [ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verification](https://mlsanity.com/paper/2005.07143/) (2020)
- [High Fidelity Neural Audio Compression](https://mlsanity.com/paper/2210.13438/) (2022)
- [Vector-quantized Image Modeling with Improved VQGAN](https://mlsanity.com/paper/2110.04627/) (2021)
- [High-Fidelity Audio Compression with Improved RVQGAN](https://mlsanity.com/paper/2306.06546/) (2023)
- [SpeechBrain: A General-Purpose Speech Toolkit](https://mlsanity.com/paper/2106.04624/) (2021)
- [CAM++: A Fast and Efficient Network for Speaker Verification Using Context-Aware Masking](https://mlsanity.com/paper/2303.00332/) (2023)
- [Neural Networks Fail to Learn Periodic Functions and How to Fix It](https://mlsanity.com/paper/2006.08195/) (2020)
- [An Enhanced Res2Net with Local and Global Feature Fusion for Speaker Verification](https://mlsanity.com/paper/2305.12838/) (2023)
- [Codec-SUPERB: An In-Depth Analysis of Sound Codec Models](https://mlsanity.com/paper/2402.13071/) (2024)
- [DASB - Discrete Audio and Speech Benchmark](https://mlsanity.com/paper/2406.14294/) (2024)
- [3D-Speaker-Toolkit: An Open-Source Toolkit for Multimodal Speaker Verification and Diarization](https://mlsanity.com/paper/2403.19971/) (2024)
- [High-Quality, Low-Delay Music Coding in the Opus Codec](https://mlsanity.com/paper/1602.04845/) (2016)
- [Speaker anonymization using neural audio codec language models](https://mlsanity.com/paper/2309.14129/) (2023)
- [Discrete Audio Representation as an Alternative to Mel-Spectrograms for Speaker and Speech Recognition](https://mlsanity.com/paper/2309.10922/) (2023)
- [Representation Collapsing Problems in Vector Quantization](https://mlsanity.com/paper/2411.16550/) (2024)
- [Neural Codec-based Adversarial Sample Detection for Speaker Verification](https://mlsanity.com/paper/2406.04582/) (2024)
- [Optimizing Neural Speech Codec for Low-Bitrate Compression via Multi-Scale Encoding](https://mlsanity.com/paper/2410.15749/) (2024)

## Abstract (arXiv)

Neural audio codecs (NACs) have made significant advancements in recent years and are rapidly being adopted in many audio processing pipelines. However, they can introduce audio distortions which degrade speaker verification (SV) performance. This study investigates the impact of both traditional and neural audio codecs at varying bitrates on three state of-the-art SV models evaluated on the VoxCeleb1 dataset. Our findings reveal a consistent degradation in SV performance across all models and codecs as bitrates decrease. Notably, NACs do not fundamentally break SV performance when compared to traditional codecs. They outperform Opus by 6-8% at low-bitrates (< 12 kbps) and remain marginally behind at higher bitrates (≈ 24 kbps), with an EER increase of only 0.4-0.7%. The disparity at higher bitrates is likely due to the primary optimization of NACs for perceptual quality, which can inadvertently discard critical speaker-discriminative features, unlike Opus which was designed to preserve vocal characteristics. Our investigation suggests that NACs are a feasible alternative to traditional codecs, especially under bandwidth limitations. To bridge the gap at higher bitrates, future work should focus on developing speaker-aware NACs or retraining and adapting SV models.