# Speaker anonymization using orthogonal Householder neural network

arXiv: https://arxiv.org/abs/2305.18823 · HTML: https://mlsanity.com/paper/2305.18823/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 May 2023.

## Built on by (most cited first, 18 of 18)

- [Privacy versus Emotion Preservation Trade-offs in Emotion-Preserving Speaker Anonymization](https://mlsanity.com/paper/2409.03655/) (2024)
- [The Third VoicePrivacy Challenge: Preserving Emotional Expressiveness and Linguistic Content in Voice Anonymization](https://mlsanity.com/paper/2601.11846/) (2026)
- [A Benchmark for Multi-speaker Anonymization](https://mlsanity.com/paper/2407.05608/) (2024)
- [DarkStream: real-time speech anonymization with low latency](https://mlsanity.com/paper/2509.04667/) (2025)
- [Stream-Voice-Anon: Enhancing Utility of Real-Time Speaker Anonymization via Neural Audio Codec and Language Models](https://mlsanity.com/paper/2601.13948/) (2026)
- [SafeSpeech: Robust and Universal Voice Protection Against Malicious Speech Synthesis](https://mlsanity.com/paper/2504.09839/) (2025)
- [Probing the Feasibility of Multilingual Speaker Anonymization](https://mlsanity.com/paper/2407.02937/) (2024)
- [Analysis of Speech Temporal Dynamics in the Context of Speaker Verification and Voice Anonymization](https://mlsanity.com/paper/2412.17164/) (2024)
- [Exploiting Context-dependent Duration Features for Voice Anonymization Attack Systems](https://mlsanity.com/paper/2507.15214/) (2025)
- [Towards Privacy-aware Mental Health AI Models: Advances, Challenges, and Opportunities](https://mlsanity.com/paper/2502.00451/) (2025)
- [SEF-MK: Speaker-Embedding-Free Voice Anonymization through Multi-k-means Quantization](https://mlsanity.com/paper/2508.07086/) (2025)
- [SegReConcat: A Data Augmentation Method for Voice Anonymization Attack](https://mlsanity.com/paper/2508.18907/) (2025)
- [Target speaker anonymization in multi-speaker recordings](https://mlsanity.com/paper/2510.09307/) (2025)
- [DAST: A Dual-Stream Voice Anonymization Attacker with Staged Training](https://mlsanity.com/paper/2603.12840/) (2026)
- [Anonymization, Not Elimination: Utility-Preserved Speech Anonymization](https://mlsanity.com/paper/2604.17000/) (2026)
- [Novel Loss-Enhanced Universal Adversarial Patches for Sustainable Speaker Privacy](https://mlsanity.com/paper/2505.19951/) (2025)
- [Dual-Granularity Orthogonal Disentanglement for Generalizable Audio Deepfake Detection](https://mlsanity.com/paper/2606.16532/) (2026)
- [NouveauVoice: Generating Novel Pseudo Speakers for Voice Anonymization](https://mlsanity.com/paper/2607.03985/) (2026)

## Abstract (arXiv)

Speaker anonymization aims to conceal a speaker's identity while preserving content information in speech. Current mainstream neural-network speaker anonymization systems disentangle speech into prosody-related, content, and speaker representations. The speaker representation is then anonymized by a selection-based speaker anonymizer that uses a mean vector over a set of randomly selected speaker vectors from an external pool of English speakers. However, the resulting anonymized vectors are subject to severe privacy leakage against powerful attackers, reduction in speaker diversity, and language mismatch problems for unseen-language speaker anonymization. To generate diverse, language-neutral speaker vectors, this paper proposes an anonymizer based on an orthogonal Householder neural network (OHNN). Specifically, the OHNN acts like a rotation to transform the original speaker vectors into anonymized speaker vectors, which are constrained to follow the distribution over the original speaker vector space. A basic classification loss is introduced to ensure that anonymized speaker vectors from different speakers have unique speaker identities. To further protect speaker identities, an improved classification loss and similarity loss are used to push original-anonymized sample pairs away from each other. Experiments on VoicePrivacy Challenge datasets in English and the textitAISHELL-3 dataset in Mandarin demonstrate the proposed anonymizer's effectiveness.