mlsanityData to 14 Oct 2026

Privacy attacks for automatic speech recognition acoustic models in a federated learning framework

arXiv 2111.03777 ↗First posted 6 Nov 2021cs.CL cs.CR cs.SDcited by 10

Not on the map. Why?

Abstract (arXiv)

This paper investigates methods to effectively retrieve speaker information from the personalized speaker adapted neural network acoustic models (AMs) in automatic speech recognition (ASR). This problem is especially important in the context of federated learning of ASR acoustic models where a global model is learnt on the server based on the updates received from multiple clients. We propose an approach to analyze information in neural network AMs based on a neural network footprint on the so-called Indicator dataset. Using this method, we develop two attack models that aim to infer speaker identity from the updated personalized models without access to the actual users' speech data. Experiments on the TED-LIUM 3 corpus demonstrate that the proposed approaches are very effective and can provide equal error rate (EER) of 1-2%.

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Builds on 17 of its 32 references that are in the map. First posted on arXiv 6 Nov 2021.

Builds on

17 of 32 references.

References · 17

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  2. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,538
  3. 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 864
  4. 2020Inverting Gradients -- How easy is it to break privacy in federated learning?cited by 712
  5. 2018Beyond Inferring Class Representatives: User-Level Privacy Leakage From Federated Learningcited by 246
  6. 2018Differentially Private Generative Adversarial Networkcited by 235
  7. 2018TED-LIUM 3: twice as much data and corpus repartition for experiments on speaker adaptationcited by 226
  8. 2016Practical Secure Aggregation for Federated Learning on User-Held Datacited by 178
  9. 2018Federated Learning for Keyword Spottingcited by 95
  10. 2020Introducing the VoicePrivacy Initiativecited by 49
  11. 2020Training Speech Recognition Models with Federated Learning: A Quality/Cost Frameworkcited by 46
  12. 2021The VoicePrivacy 2020 Challenge: Results and findingscited by 46
5 more
  1. 2020Training Keyword Spotting Models on Non-IID Data with Federated Learningcited by 36
  2. 2020Improving on-device speaker verification using federated learning with privacycited by 30
  3. 2021Federated Acoustic Modeling For Automatic Speech Recognitioncited by 20
  4. 2021Federated Learning in ASR: Not as Easy as You Thinkcited by 14
  5. 2020Federated Transfer Learning with Dynamic Gradient Aggregationcited by 8

Built on by

Cited by year

Citing papers per year: 2021: 1; 2022: 3; 2023: 2; 2024: 2; 2025: 2.

10 papers.

  1. 2024Automatic Speech Recognition using Advanced Deep Learning Approaches: A surveycited by 50
  2. 2022A Review of Speech-centric Trustworthy Machine Learning: Privacy, Safety, and Fairnesscited by 13
  3. 2023Federated Learning for ASR based on Wav2vec 2.0cited by 7
  4. 2022Robust Federated Learning Against Adversarial Attacks for Speech Emotion Recognitioncited by 6
  5. 2022Decoupled Federated Learning for ASR with Non-IID Datacited by 6
  6. 2021Retrieving Speaker Information from Personalized Acoustic Models for Speech Recognitioncited by 2
  7. 2023FedAVO: Improving Communication Efficiency in Federated Learning with African Vultures Optimizercited by 2
  8. 2025Towards Privacy-aware Mental Health AI Models: Advances, Challenges, and Opportunitiescited by 2
  9. 2024Federating Dynamic Models using Early-Exit Architectures for Automatic Speech Recognition on Heterogeneous Clientscited by 1
  10. 2025Personal Attribute Leakage in Federated Speech Modelscited by 0