On-Device Speaker Anonymization of Acoustic Embeddings for ASR based onFlexible Location Gradient Reversal Layer
On the map: Speech and audio language models → Anti-spoofing and adversarial audio attacks
Abstract (arXiv)
Smart devices serviced by large-scale AI models necessitates user data transfer to the cloud for inference. For speech applications, this means transferring private user information, e.g., speaker identity. Our paper proposes a privacy-enhancing framework that targets speaker identity anonymization while preserving speech recognition accuracy for our downstream task~-~Automatic Speech Recognition (ASR). The proposed framework attaches flexible gradient reversal based speaker adversarial layers to target layers within an ASR model, where speaker adversarial training anonymizes acoustic embeddings generated by the targeted layers to remove speaker identity. We propose on-device deployment by execution of initial layers of the ASR model, and transmitting anonymized embeddings to the cloud, where the rest of the model is executed while preserving privacy. Experimental results show that our method efficiently reduces speaker recognition relative accuracy by 33%, and improves ASR performance by achieving 6.2% relative Word Error Rate (WER) reduction.
Key facts. No citing papers in the map yet (data through 19 Oct 2026). Placed in Anti-spoofing and adversarial audio attacks, one of 11 directions in Speech and audio language models. Builds on 21 of its 29 references that are in the map. First posted on arXiv 25 Jul 2023.
Builds on
21 of 29 references.
Closest ancestors · 1
Same direction, newest first
Other references · 20
- 2014Explaining and Harnessing Adversarial Examplescited by 7,467
- 2013Intriguing properties of neural networkscited by 5,227
- 2015Domain-Adversarial Training of Neural Networkscited by 3,077
- 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 2,293
- 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,462
- 2018VoxCeleb2: Deep Speaker Recognitioncited by 1,166
- 2017VoxCeleb: a large-scale speaker identification datasetcited by 985
- 2021SpeechBrain: A General-Purpose Speech Toolkitcited by 532
- 2019MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesiscited by 436
- 2018Distributed learning of deep neural network over multiple agentscited by 280
- 2017State-of-the-art Speech Recognition With Sequence-to-Sequence Modelscited by 224
- 2019Evaluating Voice Conversion-based Privacy Protection against Informed Attackerscited by 33
8 more
- 2019Privacy-Preserving Adversarial Representation Learning in ASR: Reality or Illusion?cited by 26
- 2019Privacy Enhanced Multimodal Neural Representations for Emotion Recognitioncited by 24
- 2022Anonymizing Speech with Generative Adversarial Networks to Preserve Speaker Privacycited by 24
- 2022Differentially Private Speaker Anonymizationcited by 21
- 2020Voice-Indistinguishability: Protecting Voiceprint in Privacy-Preserving Speech Data Releasecited by 13
- 2022Enhancing and Adversarial: Improve ASR with Speaker Labelscited by 7
- 2022Insights on Neural Representations for End-to-End Speech Recognitioncited by 6
- 2022Probing Statistical Representations For End-To-End ASRcited by 2
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Towards Privacy-Preserving ASR: Speaker-Level Machine Unlearning
- 2025Whispering Under the Eaves: Protecting User Privacy Against Commercial and LLM-powered Automatic Speech Recognition Systems
- 2025Private kNN-VC: Interpretable Anonymization of Converted Speech
- 2022A Speech Representation Anonymization Framework via Selective Noise Perturbation
- 2022Distinguishable Speaker Anonymization based on Formant and Fundamental Frequency Scaling
- 2025Inference Attacks for X-Vector Speaker Anonymization
- 2023Privacy-oriented manipulation of speaker representations
- 2024Distinctive and Natural Speaker Anonymization via Singular Value Transformation-assisted Matrix