# Speaker anonymization using neural audio codec language models

arXiv: https://arxiv.org/abs/2309.14129 · HTML: https://mlsanity.com/paper/2309.14129/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Sep 2023.

## Built on by (most cited first, 22 of 22)

- [The VoicePrivacy 2024 Challenge Evaluation Plan](https://mlsanity.com/paper/2404.02677/) (2024)
- [A Benchmark for Multi-speaker Anonymization](https://mlsanity.com/paper/2407.05608/) (2024)
- [Stream-Voice-Anon: Enhancing Utility of Real-Time Speaker Anonymization via Neural Audio Codec and Language Models](https://mlsanity.com/paper/2601.13948/) (2026)
- [Analysis of Speech Temporal Dynamics in the Context of Speaker Verification and Voice Anonymization](https://mlsanity.com/paper/2412.17164/) (2024)
- [Children's Voice Privacy: First Steps And Emerging Challenges](https://mlsanity.com/paper/2506.00100/) (2025)
- [The First VoicePrivacy Attacker Challenge](https://mlsanity.com/paper/2504.14183/) (2025)
- [Exploiting Context-dependent Duration Features for Voice Anonymization Attack Systems](https://mlsanity.com/paper/2507.15214/) (2025)
- [Exploring VQ-VAE with Prosody Parameters for Speaker Anonymization](https://mlsanity.com/paper/2409.15882/) (2024)
- [You Are What You Say: Exploiting Linguistic Content for VoicePrivacy Attacks](https://mlsanity.com/paper/2506.09521/) (2025)
- [The Impact of Automatic Speech Transcription on Speaker Attribution](https://mlsanity.com/paper/2507.08660/) (2025)
- [Speech After Gender: A Trans-Feminine Perspective on Next Steps for Speech Science and Technology](https://mlsanity.com/paper/2407.07235/) (2024)
- [The First VoicePrivacy Attacker Challenge Evaluation Plan](https://mlsanity.com/paper/2410.07428/) (2024)
- [SEF-MK: Speaker-Embedding-Free Voice Anonymization through Multi-k-means Quantization](https://mlsanity.com/paper/2508.07086/) (2025)
- [SegReConcat: A Data Augmentation Method for Voice Anonymization Attack](https://mlsanity.com/paper/2508.18907/) (2025)
- [StreamVoiceAnon+: Emotion-Preserving Streaming Speaker Anonymization via Frame-Level Acoustic Distillation](https://mlsanity.com/paper/2603.06079/) (2026)
- [Privacy-Preserving End-to-End Full-Duplex Speech Dialogue Models](https://mlsanity.com/paper/2603.08179/) (2026)
- [Anonymization, Not Elimination: Utility-Preserved Speech Anonymization](https://mlsanity.com/paper/2604.17000/) (2026)
- [Perceptual implications of automatic anonymization in pathological speech](https://mlsanity.com/paper/2505.00409/) (2025)
- [Voice Privacy from an Attribute-based Perspective](https://mlsanity.com/paper/2603.20301/) (2026)
- [Child-Centric Voice Anonymization in Single and Multi-Speaker Speech via Domain-Adapted SSL Models](https://mlsanity.com/paper/2606.29897/) (2026)
- [Your Voice Cloning System is Secretly a Voice Anonymizer](https://mlsanity.com/paper/2608.27360/) (2026)
- [LiveAnon: Improving Privacy and Utility in Streaming Speaker Anonymization with Codec Language Models](https://mlsanity.com/paper/2610.15379/) (2026)

## Abstract (arXiv)

The vast majority of approaches to speaker anonymization involve the extraction of fundamental frequency estimates, linguistic features and a speaker embedding which is perturbed to obfuscate the speaker identity before an anonymized speech waveform is resynthesized using a vocoder. Recent work has shown that x-vector transformations are difficult to control consistently: other sources of speaker information contained within fundamental frequency and linguistic features are re-entangled upon vocoding, meaning that anonymized speech signals still contain speaker information. We propose an approach based upon neural audio codecs (NACs), which are known to generate high-quality synthetic speech when combined with language models. NACs use quantized codes, which are known to effectively bottleneck speaker-related information: we demonstrate the potential of speaker anonymization systems based on NAC language modeling by applying the evaluation framework of the Voice Privacy Challenge 2022.