# VoxAnon: A balanced dataset for a fair evaluation of speech anonymization

arXiv: https://arxiv.org/abs/2610.13313 · HTML: https://mlsanity.com/paper/2610.13313/

## Key facts

- No citing papers in the map yet (data through 19 Oct 2026).
- Placed in Anti-spoofing and adversarial audio attacks, one of 11 directions in Speech and audio language models.
- Builds on 25 of its 31 references that are in the map.
- First posted on arXiv 9 Oct 2026.

## Placed in

- Speech and audio language models / [Anti-spoofing and adversarial audio attacks](https://mlsanity.com/speech/anti-spoofing-and-adversarial-audio-attacks/)

## Builds on (most cited first, 25 of 25)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing](https://mlsanity.com/paper/2110.13900/) (2021)
- [VoxCeleb2: Deep Speaker Recognition](https://mlsanity.com/paper/1806.05622/) (2018)
- [VoxCeleb: a large-scale speaker identification dataset](https://mlsanity.com/paper/1706.08612/) (2017)
- [ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verification](https://mlsanity.com/paper/2005.07143/) (2020)
- [SONAR: Sentence-Level Multimodal and Language-Agnostic Representations](https://mlsanity.com/paper/2308.11466/) (2023)
- [The VoicePrivacy 2020 Challenge: Results and findings](https://mlsanity.com/paper/2109.00648/) (2021)
- [Reshape Dimensions Network for Speaker Recognition](https://mlsanity.com/paper/2407.18223/) (2024)
- [Evaluating Voice Conversion-based Privacy Protection against Informed Attackers](https://mlsanity.com/paper/1911.03934/) (2019)
- [VoxCeleb Enrichment for Age and Gender Recognition](https://mlsanity.com/paper/2109.13510/) (2021)
- [Anonymizing Speech: Evaluating and Designing Speaker Anonymization Techniques](https://mlsanity.com/paper/2308.04455/) (2023)
- [Speaker anonymization using neural audio codec language models](https://mlsanity.com/paper/2309.14129/) (2023)
- [To train or not to train adversarially: A study of bias mitigation strategies for speaker recognition](https://mlsanity.com/paper/2203.09122/) (2022)
- [SVeritas: Benchmark for Robust Speaker Verification under Diverse Conditions](https://mlsanity.com/paper/2509.17091/) (2025)
- [Improving Fairness in Speaker Recognition](https://mlsanity.com/paper/2104.14067/) (2021)
- [You Are What You Say: Exploiting Linguistic Content for VoicePrivacy Attacks](https://mlsanity.com/paper/2506.09521/) (2025)
- [C-P Map: A Novel Evaluation Toolkit for Speaker Verification](https://mlsanity.com/paper/2203.02942/) (2022)
- [Voice Anonymization for All -- Bias Evaluation of the Voice Privacy Challenge Baseline System](https://mlsanity.com/paper/2311.15804/) (2023)
- [A Comparison of Differential Performance Metrics for the Evaluation of Automatic Speaker Verification Fairness](https://mlsanity.com/paper/2404.17810/) (2024)
- [Content Leakage in LibriSpeech and Its Impact on the Privacy Evaluation of Speaker Anonymization](https://mlsanity.com/paper/2601.13107/) (2026)
- [DAST: A Dual-Stream Voice Anonymization Attacker with Staged Training](https://mlsanity.com/paper/2603.12840/) (2026)
- [SpeakerCard-1M: An Evidence-Grounded Corpus for In-the-Wild Speaker Verification](https://mlsanity.com/paper/2606.03283/) (2026)
- [Kiwano: A Cutting-Edge Open-Source Toolkit for Speaker Verification](https://mlsanity.com/paper/2606.22369/) (2026)
- [Trade-offs Between Individual and Group Fairness in Machine Learning: A Comprehensive Review](https://mlsanity.com/paper/2602.00094/) (2026)
- [A Large-Scale Per-Speaker Analysis of Re-identification Risk in Speech Anonymization](https://mlsanity.com/paper/2606.07210/) (2026)

## Abstract (arXiv)

Speech anonymization is evaluated with automatic speaker verification attackers whose performance depends on the attribute distribution of both their training and evaluation data, which may bias privacy estimates for some speaker groups. To alleviate this issue, we introduce VoxAnon, a dataset built from VoxCeleb1 and 2 with training and evaluation splits balanced across gender, age, accent, and pitch. We extract and cross-check attributes using two complementary methods, balance trial lists per subgroup, and draw test/enrollment utterances from distinct video sources in order to limit linguistic and session bias. Comparing VoxAnon with unbalanced training and test data across four attackers and three anonymizers, we show that attribute imbalance changes which subgroups appear hardest to re-identify, in directions that depend on the attribute. In the balanced setting, residual disparities persist, mostly tied to accent and pitch, and depend on the anonymizer/attacker pair. Code and trial lists are publicly available.