# SEF-MK: Speaker-Embedding-Free Voice Anonymization through Multi-k-means Quantization

arXiv: https://arxiv.org/abs/2508.07086 · HTML: https://mlsanity.com/paper/2508.07086/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 50 references that are in the map.
- First posted on arXiv 9 Aug 2025.

## Builds on (most cited first, 30 of 37)

- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units](https://mlsanity.com/paper/2106.07447/) (2021)
- [Conformer: Convolution-augmented Transformer for Speech Recognition](https://mlsanity.com/paper/2005.08100/) (2020)
- [WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing](https://mlsanity.com/paper/2110.13900/) (2021)
- [HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis](https://mlsanity.com/paper/2010.05646/) (2020)
- [VoxCeleb2: Deep Speaker Recognition](https://mlsanity.com/paper/1806.05622/) (2018)
- [SUPERB: Speech processing Universal PERformance Benchmark](https://mlsanity.com/paper/2105.01051/) (2021)
- [LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speech](https://mlsanity.com/paper/1904.02882/) (2019)
- [ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verification](https://mlsanity.com/paper/2005.07143/) (2020)
- [Voice Conversion With Just Nearest Neighbors](https://mlsanity.com/paper/2305.18975/) (2023)
- [Introducing the VoicePrivacy Initiative](https://mlsanity.com/paper/2005.01387/) (2020)
- [The VoicePrivacy 2020 Challenge: Results and findings](https://mlsanity.com/paper/2109.00648/) (2021)
- [Speaker Anonymization Using X-vector and Neural Waveform Models](https://mlsanity.com/paper/1905.13561/) (2019)
- [The VoicePrivacy 2024 Challenge Evaluation Plan](https://mlsanity.com/paper/2404.02677/) (2024)
- [Speaker anonymisation using the McAdams coefficient](https://mlsanity.com/paper/2011.01130/) (2020)
- [Evaluating Voice Conversion-based Privacy Protection against Informed Attackers](https://mlsanity.com/paper/1911.03934/) (2019)
- [Design Choices for X-vector Based Speaker Anonymization](https://mlsanity.com/paper/2005.08601/) (2020)
- [Anonymizing Speech: Evaluating and Designing Speaker Anonymization Techniques](https://mlsanity.com/paper/2308.04455/) (2023)
- [Speaker anonymization using neural audio codec language models](https://mlsanity.com/paper/2309.14129/) (2023)
- [Anonymizing Speech with Generative Adversarial Networks to Preserve Speaker Privacy](https://mlsanity.com/paper/2210.07002/) (2022)
- [The VoicePrivacy 2022 Challenge Evaluation Plan](https://mlsanity.com/paper/2203.12468/) (2022)
- [Speaker anonymization using orthogonal Householder neural network](https://mlsanity.com/paper/2305.18823/) (2023)
- [The VoicePrivacy 2022 Challenge: Progress and Perspectives in Voice Anonymisation](https://mlsanity.com/paper/2407.11516/) (2024)
- [Differentially Private Speaker Anonymization](https://mlsanity.com/paper/2202.11823/) (2022)
- [Language-Independent Speaker Anonymization Approach using Self-Supervised Pre-Trained Models](https://mlsanity.com/paper/2202.13097/) (2022)
- [Improving speaker de-identification with functional data analysis of f0 trajectories](https://mlsanity.com/paper/2203.16738/) (2022)
- [MUSA: Multi-lingual Speaker Anonymization via Serial Disentanglement](https://mlsanity.com/paper/2407.11629/) (2024)
- [Privacy versus Emotion Preservation Trade-offs in Emotion-Preserving Speaker Anonymization](https://mlsanity.com/paper/2409.03655/) (2024)
- [HLTCOE JHU Submission to the Voice Privacy Challenge 2024](https://mlsanity.com/paper/2409.08913/) (2024)
- [A Benchmark for Multi-speaker Anonymization](https://mlsanity.com/paper/2407.05608/) (2024)

## Built on by (most cited first, 1 of 1)

- [The Third VoicePrivacy Challenge: Preserving Emotional Expressiveness and Linguistic Content in Voice Anonymization](https://mlsanity.com/paper/2601.11846/) (2026)

## Abstract (arXiv)

Voice anonymization protects speaker privacy by concealing identity while preserving linguistic and paralinguistic content. Self-supervised learning (SSL) representations encode linguistic features but preserve speaker traits. We propose a novel speaker-embedding-free framework called SEF-MK. Instead of using a single k-means model trained on the entire dataset, SEF-MK anonymizes SSL representations for each utterance by randomly selecting one of multiple k-means models, each trained on a different subset of speakers. We explore this approach from both attacker and user perspectives. Extensive experiments show that, compared to a single k-means model, SEF-MK with multiple k-means models better preserves linguistic and emotional content from the user's viewpoint. However, from the attacker's perspective, utilizing multiple k-means models boosts the effectiveness of privacy attacks. These insights can aid users in designing voice anonymization systems to mitigate attacker threats.