# Privacy versus Emotion Preservation Trade-offs in Emotion-Preserving Speaker Anonymization

arXiv: https://arxiv.org/abs/2409.03655 · HTML: https://mlsanity.com/paper/2409.03655/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 30 references that are in the map.
- First posted on arXiv 5 Sep 2024.

## Builds on (most cited first, 19 of 19)

- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing](https://mlsanity.com/paper/2110.13900/) (2021)
- [HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis](https://mlsanity.com/paper/2010.05646/) (2020)
- [VoxCeleb: a large-scale speaker identification dataset](https://mlsanity.com/paper/1706.08612/) (2017)
- [LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speech](https://mlsanity.com/paper/1904.02882/) (2019)
- [ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verification](https://mlsanity.com/paper/2005.07143/) (2020)
- [SpeechBrain: A General-Purpose Speech Toolkit](https://mlsanity.com/paper/2106.04624/) (2021)
- [Dawn of the transformer era in speech emotion recognition: closing the valence gap](https://mlsanity.com/paper/2203.07378/) (2022)
- [ContentVec: An Improved Self-Supervised Speech Representation by Disentangling Speakers](https://mlsanity.com/paper/2204.09224/) (2022)
- [Voice Conversion With Just Nearest Neighbors](https://mlsanity.com/paper/2305.18975/) (2023)
- [Introducing the VoicePrivacy Initiative](https://mlsanity.com/paper/2005.01387/) (2020)
- [Speaker Anonymization Using X-vector and Neural Waveform Models](https://mlsanity.com/paper/1905.13561/) (2019)
- [The VoicePrivacy 2024 Challenge Evaluation Plan](https://mlsanity.com/paper/2404.02677/) (2024)
- [The VoicePrivacy 2022 Challenge Evaluation Plan](https://mlsanity.com/paper/2203.12468/) (2022)
- [Speaker anonymization using orthogonal Householder neural network](https://mlsanity.com/paper/2305.18823/) (2023)
- [Disentanglement of Emotional Style and Speaker Identity for Expressive Voice Conversion](https://mlsanity.com/paper/2110.10326/) (2021)
- [Speaker Anonymization with Phonetic Intermediate Representations](https://mlsanity.com/paper/2207.04834/) (2022)
- [Speaker Anonymization with Distribution-Preserving X-Vector Generation for the VoicePrivacy Challenge 2020](https://mlsanity.com/paper/2010.13457/) (2020)
- [Identifying Source Speakers for Voice Conversion based Spoofing Attacks on Speaker Verification Systems](https://mlsanity.com/paper/2206.09103/) (2022)

## Built on by (most cited first, 10 of 10)

- [GenVC: Self-Supervised Zero-Shot Voice Conversion](https://mlsanity.com/paper/2502.04519/) (2025)
- [The Third VoicePrivacy Challenge: Preserving Emotional Expressiveness and Linguistic Content in Voice Anonymization](https://mlsanity.com/paper/2601.11846/) (2026)
- [The Impact of Automatic Speech Transcription on Speaker Attribution](https://mlsanity.com/paper/2507.08660/) (2025)
- [Universal Semantic Disentangled Privacy-preserving Speech Representation Learning](https://mlsanity.com/paper/2505.13085/) (2025)
- [SEF-MK: Speaker-Embedding-Free Voice Anonymization through Multi-k-means Quantization](https://mlsanity.com/paper/2508.07086/) (2025)
- [SegReConcat: A Data Augmentation Method for Voice Anonymization Attack](https://mlsanity.com/paper/2508.18907/) (2025)
- [StreamVoiceAnon+: Emotion-Preserving Streaming Speaker Anonymization via Frame-Level Acoustic Distillation](https://mlsanity.com/paper/2603.06079/) (2026)
- [DAST: A Dual-Stream Voice Anonymization Attacker with Staged Training](https://mlsanity.com/paper/2603.12840/) (2026)
- [DiffAnon: Diffusion-based Prosody Control for Voice Anonymization](https://mlsanity.com/paper/2604.26281/) (2026)
- [LiveAnon: Improving Privacy and Utility in Streaming Speaker Anonymization with Codec Language Models](https://mlsanity.com/paper/2610.15379/) (2026)

## Abstract (arXiv)

Advances in speech technology now allow unprecedented access to personally identifiable information through speech. To protect such information, the differential privacy field has explored ways to anonymize speech while preserving its utility, including linguistic and paralinguistic aspects. However, anonymizing speech while maintaining emotional state remains challenging. We explore this problem in the context of the VoicePrivacy 2024 challenge. Specifically, we developed various speaker anonymization pipelines and find that approaches either excel at anonymization or preserving emotion state, but not both simultaneously. Achieving both would require an in-domain emotion recognizer. Additionally, we found that it is feasible to train a semi-effective speaker verification system using only emotion representations, demonstrating the challenge of separating these two modalities.