# You Are What You Say: Exploiting Linguistic Content for VoicePrivacy Attacks

arXiv: https://arxiv.org/abs/2506.09521 · HTML: https://mlsanity.com/paper/2506.09521/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 24 references that are in the map.
- First posted on arXiv 11 Jun 2025.

## Builds on (most cited first, 15 of 15)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verification](https://mlsanity.com/paper/2005.07143/) (2020)
- [Open-Source Conversational AI with SpeechBrain 1.0](https://mlsanity.com/paper/2407.00463/) (2024)
- [Introducing the VoicePrivacy Initiative](https://mlsanity.com/paper/2005.01387/) (2020)
- [Speaker anonymisation using the McAdams coefficient](https://mlsanity.com/paper/2011.01130/) (2020)
- [Evaluating Voice Conversion-based Privacy Protection against Informed Attackers](https://mlsanity.com/paper/1911.03934/) (2019)
- [Margin Matters: Towards More Discriminative Deep Neural Network Embeddings for Speaker Recognition](https://mlsanity.com/paper/1906.07317/) (2019)
- [Anonymizing Speech: Evaluating and Designing Speaker Anonymization Techniques](https://mlsanity.com/paper/2308.04455/) (2023)
- [Speaker anonymization using neural audio codec language models](https://mlsanity.com/paper/2309.14129/) (2023)
- [The VoicePrivacy 2022 Challenge: Progress and Perspectives in Voice Anonymisation](https://mlsanity.com/paper/2407.11516/) (2024)
- [The Impact of Speech Anonymization on Pathology and Its Limits](https://mlsanity.com/paper/2404.08064/) (2024)
- [The Privacy ZEBRA: Zero Evidence Biometric Recognition Assessment](https://mlsanity.com/paper/2005.09413/) (2020)
- [Attacking Voice Anonymization Systems with Augmented Feature and Speaker Identity Difference](https://mlsanity.com/paper/2412.19068/) (2024)
- [On the invertibility of a voice privacy system using embedding alignement](https://mlsanity.com/paper/2110.05431/) (2021)
- [Evaluating X-vector-based Speaker Anonymization under White-box Assessment](https://mlsanity.com/paper/2109.11946/) (2021)

## Built on by (most cited first, 2 of 2)

- [Anonymization, Not Elimination: Utility-Preserved Speech Anonymization](https://mlsanity.com/paper/2604.17000/) (2026)
- [Exploiting Acoustic and Content-Oriented Speaker Verification Attacks Against Multilingual Voice Anonymization](https://mlsanity.com/paper/2610.08107/) (2026)

## Abstract (arXiv)

Speaker anonymization systems hide the identity of speakers while preserving other information such as linguistic content and emotions. To evaluate their privacy benefits, attacks in the form of automatic speaker verification (ASV) systems are employed. In this study, we assess the impact of intra-speaker linguistic content similarity in the attacker training and evaluation datasets, by adapting BERT, a language model, as an ASV system. On the VoicePrivacy Attacker Challenge datasets, our method achieves a mean equal error rate (EER) of 35%, with certain speakers attaining EERs as low as 2%, based solely on the textual content of their utterances. Our explainability study reveals that the system decisions are linked to semantically similar keywords within utterances, stemming from how LibriSpeech is curated. Our study suggests reworking the VoicePrivacy datasets to ensure a fair and unbiased evaluation and challenge the reliance on global EER for privacy evaluations.