mlsanityData to 7 Oct 2026

Improving on-device speaker verification using federated learning with privacy

arXiv 2008.02651 ↗First posted 6 Aug 2020eess.AS cs.LG cs.SDcited by 29

Not on the map. Why?

Abstract (arXiv)

Information on speaker characteristics can be useful as side information in improving speaker recognition accuracy. However, such information is often private. This paper investigates how privacy-preserving learning can improve a speaker verification system, by enabling the use of privacy-sensitive speaker data to train an auxiliary classification model that predicts vocal characteristics of speakers. In particular, this paper explores the utility achieved by approaches which combine different federated learning and differential privacy mechanisms. These approaches make it possible to train a central model while protecting user privacy, with users' data remaining on their devices. Furthermore, they make learning on a large population of speakers possible, ensuring good coverage of speaker characteristics when training a model. The auxiliary model described here uses features extracted from phrases which trigger a speaker verification system. From these features, the model predicts speaker characteristic labels considered useful as side information. The knowledge of the auxiliary model is distilled into a speaker verification system using multi-task learning, with the side information labels predicted by this auxiliary model being the additional task. This approach results in a 6% relative improvement in equal error rate over a baseline system.

Key facts. Cited by 29 papers in the map (data through 7 Oct 2026). First posted on arXiv 6 Aug 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 7; 2022: 8; 2023: 6; 2024: 3; 2025: 3; 2026: 2.

29 papers.

Sort:

  1. 2021Federated Evaluation and Tuning for On-Device Personalization: System Design & Applicationscited by 84
  2. 2022Auditing Privacy Defenses in Federated Learning via Generative Gradient Leakagecited by 72
  3. 2021Federated Learning for Internet of Things: Applications, Challenges, and Opportunitiescited by 49
  4. 2021Enforcing fairness in private federated learning via the modified method of differential multiplierscited by 38
  5. 2021End-to-End Speech Recognition from Federated Acoustic Modelscited by 30
  6. 2023Mobile Foundation Model as Firmwarecited by 25
  7. 2022FLAIR: Federated Learning Annotated Image Repositorycited by 22
  8. 2021Efficient and Private Federated Learning with Partially Trainable Networkscited by 13
  9. 2022A Review of Speech-centric Trustworthy Machine Learning: Privacy, Safety, and Fairnesscited by 13
  10. 2021Federated Learning of User Verification Models Without Sharing Embeddingscited by 11
  11. 2021Privacy attacks for automatic speech recognition acoustic models in a federated learning frameworkcited by 10
  12. 2022Federated Self-supervised Speech Representations: Are We There Yet?cited by 10
17 more
  1. 2022Production federated keyword spotting via distillation, filtering, and joint federated-centralized trainingcited by 9
  2. 2023Backdoor Attacks in Peer-to-Peer Federated Learningcited by 9
  3. 2024pfl-research: simulation framework for accelerating research in Private Federated Learningcited by 8
  4. 2023Federated Learning for ASR based on Wav2vec 2.0cited by 7
  5. 2023Samplable Anonymous Aggregation for Private Federated Data Analysiscited by 6
  6. 2024GLOCALFAIR: Jointly Improving Global and Local Group Fairness in Federated Learningcited by 5
  7. 2022ILASR: Privacy-Preserving Incremental Learning for Automatic Speech Recognition at Production Scalecited by 4
  8. 2022"You Can't Fix What You Can't Measure": Privately Measuring Demographic Performance Disparities in Federated Learningcited by 3
  9. 2024Improved Modelling of Federated Datasets using Mixtures-of-Dirichlet-Multinomialscited by 3
  10. 2022Adversarial Speaker Distillation for Countermeasure Model on Automatic Speaker Verificationcited by 2
  11. 2025Safeguarding Privacy in Edge Speech Understanding with Tiny Foundation Modelscited by 2
  12. 2023Speech Privacy Leakage from Shared Gradients in Distributed Learningcited by 1
  13. 2025Regularized Federated Learning for Privacy-Preserving Dysarthric and Elderly Speech Recognitioncited by 1
  14. 2023Federated Representation Learning for Automatic Speech Recognitioncited by 0
  15. 2025LAPA-based Dynamic Privacy Optimization for Wireless Federated Learning in Heterogeneous Environmentscited by 0
  16. 2026Collaborative Threshold Watermarkingcited by 0
  17. 2026Towards Personalized Federated Learning for Dysarthric Speech Recognitioncited by 0