# Anonymizing Speech: Evaluating and Designing Speaker Anonymization Techniques

arXiv: https://arxiv.org/abs/2308.04455 · HTML: https://mlsanity.com/paper/2308.04455/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Aug 2023.

## Built on by (most cited first, 22 of 22)

- [The VoicePrivacy 2024 Challenge Evaluation Plan](https://mlsanity.com/paper/2404.02677/) (2024)
- [The Third VoicePrivacy Challenge: Preserving Emotional Expressiveness and Linguistic Content in Voice Anonymization](https://mlsanity.com/paper/2601.11846/) (2026)
- [A Benchmark for Multi-speaker Anonymization](https://mlsanity.com/paper/2407.05608/) (2024)
- [Stream-Voice-Anon: Enhancing Utility of Real-Time Speaker Anonymization via Neural Audio Codec and Language Models](https://mlsanity.com/paper/2601.13948/) (2026)
- [NTU-NPU System for Voice Privacy 2024 Challenge](https://mlsanity.com/paper/2410.02371/) (2024)
- [Analysis of Speech Temporal Dynamics in the Context of Speaker Verification and Voice Anonymization](https://mlsanity.com/paper/2412.17164/) (2024)
- [Children's Voice Privacy: First Steps And Emerging Challenges](https://mlsanity.com/paper/2506.00100/) (2025)
- [The First VoicePrivacy Attacker Challenge](https://mlsanity.com/paper/2504.14183/) (2025)
- [The Art of Deception: Robust Backdoor Attack using Dynamic Stacking of Triggers](https://mlsanity.com/paper/2401.01537/) (2024)
- [Sound Check: Auditing Audio Datasets](https://mlsanity.com/paper/2410.13114/) (2024)
- [Exploiting Context-dependent Duration Features for Voice Anonymization Attack Systems](https://mlsanity.com/paper/2507.15214/) (2025)
- [Exploring VQ-VAE with Prosody Parameters for Speaker Anonymization](https://mlsanity.com/paper/2409.15882/) (2024)
- [You Are What You Say: Exploiting Linguistic Content for VoicePrivacy Attacks](https://mlsanity.com/paper/2506.09521/) (2025)
- [The First VoicePrivacy Attacker Challenge Evaluation Plan](https://mlsanity.com/paper/2410.07428/) (2024)
- [SEF-MK: Speaker-Embedding-Free Voice Anonymization through Multi-k-means Quantization](https://mlsanity.com/paper/2508.07086/) (2025)
- [SegReConcat: A Data Augmentation Method for Voice Anonymization Attack](https://mlsanity.com/paper/2508.18907/) (2025)
- [VoxGuard: Evaluating User and Attribute Privacy in Speech via Membership Inference Attacks](https://mlsanity.com/paper/2509.18413/) (2025)
- [Target speaker anonymization in multi-speaker recordings](https://mlsanity.com/paper/2510.09307/) (2025)
- [Anonymization, Not Elimination: Utility-Preserved Speech Anonymization](https://mlsanity.com/paper/2604.17000/) (2026)
- [Perceptual implications of automatic anonymization in pathological speech](https://mlsanity.com/paper/2505.00409/) (2025)
- [Improving the Speaker Anonymization Evaluation's Robustness to Target Speakers with Adversarial Learning](https://mlsanity.com/paper/2508.09803/) (2025)
- [Voice Privacy from an Attribute-based Perspective](https://mlsanity.com/paper/2603.20301/) (2026)

## Abstract (arXiv)

The growing use of voice user interfaces has led to a surge in the collection and storage of speech data. While data collection allows for the development of efficient tools powering most speech services, it also poses serious privacy issues for users as centralized storage makes private personal speech data vulnerable to cyber threats. With the increasing use of voice-based digital assistants like Amazon's Alexa, Google's Home, and Apple's Siri, and with the increasing ease with which personal speech data can be collected, the risk of malicious use of voice-cloning and speaker/gender/pathological/etc. recognition has increased. This thesis proposes solutions for anonymizing speech and evaluating the degree of the anonymization. In this work, anonymization refers to making personal speech data unlinkable to an identity while maintaining the usefulness (utility) of the speech signal (e.g., access to linguistic content). We start by identifying several challenges that evaluation protocols need to consider to evaluate the degree of privacy protection properly. We clarify how anonymization systems must be configured for evaluation purposes and highlight that many practical deployment configurations do not permit privacy evaluation. Furthermore, we study and examine the most common voice conversion-based anonymization system and identify its weak points before suggesting new methods to overcome some limitations. We isolate all components of the anonymization system to evaluate the degree of speaker PPI associated with each of them. Then, we propose several transformation methods for each component to reduce as much as possible speaker PPI while maintaining utility. We promote anonymization algorithms based on quantization-based transformation as an alternative to the most-used and well-known noise-based approach. Finally, we endeavor a new attack method to invert anonymization.