# On-Device Speaker Anonymization of Acoustic Embeddings for ASR based onFlexible Location Gradient Reversal Layer

arXiv: https://arxiv.org/abs/2307.13343 · HTML: https://mlsanity.com/paper/2307.13343/

## Key facts

- No citing papers in the map yet (data through 19 Oct 2026).
- Placed in Anti-spoofing and adversarial audio attacks, one of 11 directions in Speech and audio language models.
- Builds on 21 of its 29 references that are in the map.
- First posted on arXiv 25 Jul 2023.

## Placed in

- Speech and audio language models / [Anti-spoofing and adversarial audio attacks](https://mlsanity.com/speech/anti-spoofing-and-adversarial-audio-attacks/)

## Builds on (most cited first, 21 of 21)

- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [Conformer: Convolution-augmented Transformer for Speech Recognition](https://mlsanity.com/paper/2005.08100/) (2020)
- [HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis](https://mlsanity.com/paper/2010.05646/) (2020)
- [VoxCeleb2: Deep Speaker Recognition](https://mlsanity.com/paper/1806.05622/) (2018)
- [VoxCeleb: a large-scale speaker identification dataset](https://mlsanity.com/paper/1706.08612/) (2017)
- [SpeechBrain: A General-Purpose Speech Toolkit](https://mlsanity.com/paper/2106.04624/) (2021)
- [MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesis](https://mlsanity.com/paper/1910.06711/) (2019)
- [Distributed learning of deep neural network over multiple agents](https://mlsanity.com/paper/1810.06060/) (2018)
- [State-of-the-art Speech Recognition With Sequence-to-Sequence Models](https://mlsanity.com/paper/1712.01769/) (2017)
- [Evaluating Voice Conversion-based Privacy Protection against Informed Attackers](https://mlsanity.com/paper/1911.03934/) (2019)
- [Privacy-Preserving Adversarial Representation Learning in ASR: Reality or Illusion?](https://mlsanity.com/paper/1911.04913/) (2019)
- [Privacy Enhanced Multimodal Neural Representations for Emotion Recognition](https://mlsanity.com/paper/1910.13212/) (2019)
- [Anonymizing Speech with Generative Adversarial Networks to Preserve Speaker Privacy](https://mlsanity.com/paper/2210.07002/) (2022)
- [Differentially Private Speaker Anonymization](https://mlsanity.com/paper/2202.11823/) (2022)
- [Voice-Indistinguishability: Protecting Voiceprint in Privacy-Preserving Speech Data Release](https://mlsanity.com/paper/2004.07442/) (2020)
- [Speaker Anonymization with Phonetic Intermediate Representations](https://mlsanity.com/paper/2207.04834/) (2022)
- [Enhancing and Adversarial: Improve ASR with Speaker Labels](https://mlsanity.com/paper/2211.06369/) (2022)
- [Insights on Neural Representations for End-to-End Speech Recognition](https://mlsanity.com/paper/2205.09456/) (2022)
- [Probing Statistical Representations For End-To-End ASR](https://mlsanity.com/paper/2211.01993/) (2022)

## Abstract (arXiv)

Smart devices serviced by large-scale AI models necessitates user data transfer to the cloud for inference. For speech applications, this means transferring private user information, e.g., speaker identity. Our paper proposes a privacy-enhancing framework that targets speaker identity anonymization while preserving speech recognition accuracy for our downstream task~-~Automatic Speech Recognition (ASR). The proposed framework attaches flexible gradient reversal based speaker adversarial layers to target layers within an ASR model, where speaker adversarial training anonymizes acoustic embeddings generated by the targeted layers to remove speaker identity. We propose on-device deployment by execution of initial layers of the ASR model, and transmitting anonymized embeddings to the cloud, where the rest of the model is executed while preserving privacy. Experimental results show that our method efficiently reduces speaker recognition relative accuracy by 33%, and improves ASR performance by achieving 6.2% relative Word Error Rate (WER) reduction.