mlsanityData to 7 Oct 2026

Federated Learning for ASR based on Wav2vec 2.0

arXiv 2302.10790 ↗First posted 20 Feb 2023eess.AS cs.LG cs.SDcited by 7

Not on the map. Why?

Abstract (arXiv)

This paper presents a study on the use of federated learning to train an ASR model based on a wav2vec 2.0 model pre-trained by self supervision. Carried out on the well-known TED-LIUM 3 dataset, our experiments show that such a model can obtain, with no use of a language model, a word error rate of 10.92% on the official TED-LIUM 3 test set, without sharing any data from the different users. We also analyse the ASR performance for speakers depending to their participation to the federated learning. Since federated learning was first introduced for privacy purposes, we also measure its ability to protect speaker identity. To do that, we exploit an approach to analyze information contained in exchanged models based on a neural network footprint on an indicator dataset. This analysis is made layer-wise and shows which layers in an exchanged wav2vec 2.0 based model bring the speaker identity information.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 26 of its 35 references that are in the map. First posted on arXiv 20 Feb 2023.

Builds on

26 of 35 references.

References · 26

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  2. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  3. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  4. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,461
  5. 2018Federated Learning with Non-IID Datacited by 1,099
  6. 2021SUPERB: Speech processing Universal PERformance Benchmarkcited by 652
  7. 2021SpeechBrain: A General-Purpose Speech Toolkitcited by 404
  8. 2019Fair Resource Allocation in Federated Learningcited by 389
  9. 2018TED-LIUM 3: twice as much data and corpus repartition for experiments on speaker adaptationcited by 224
  10. 2017State-of-the-art Speech Recognition With Sequence-to-Sequence Modelscited by 196
  11. 2018Federated Learning for Keyword Spottingcited by 95
  12. 2020Introducing the VoicePrivacy Initiativecited by 48
14 more
  1. 2021LeBenchmark: A Reproducible Framework for Assessing Self-Supervised Representation Learning from Speechcited by 46
  2. 2021The VoicePrivacy 2020 Challenge: Results and findingscited by 45
  3. 2022Cross-Silo Federated Learning: Challenges and Opportunitiescited by 38
  4. 2020Training Keyword Spotting Models on Non-IID Data with Federated Learningcited by 36
  5. 2021End-to-End Speech Recognition from Federated Acoustic Modelscited by 30
  6. 2020Improving on-device speaker verification using federated learning with privacycited by 29
  7. 2021Federated Learning in ASR: Not as Easy as You Thinkcited by 14
  8. 2021Privacy attacks for automatic speech recognition acoustic models in a federated learning frameworkcited by 10
  9. 2022Federated Self-supervised Speech Representations: Are We There Yet?cited by 10
  10. 2022Production federated keyword spotting via distillation, filtering, and joint federated-centralized trainingcited by 9
  11. 2022Federated Domain Adaptation for ASR with Full Self-Supervisioncited by 6
  12. 2022Decoupled Federated Learning for ASR with Non-IID Datacited by 6
  13. 2022FedNST: Federated Noisy Student Training for Automatic Speech Recognitioncited by 5
  14. 2021Retrieving Speaker Information from Personalized Acoustic Models for Speech Recognitioncited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 5; 2025: 0; 2026: 2.

7 papers.

  1. 2024Automatic Speech Recognition using Advanced Deep Learning Approaches: A surveycited by 50
  2. 2024Communication-Efficient Personalized Federated Learning for Speech-to-Text Taskscited by 8
  3. 2024Federating Dynamic Models using Early-Exit Architectures for Automatic Speech Recognition on Heterogeneous Clientscited by 1
  4. 2024SVSNet+: Enhancing Speaker Voice Similarity Assessment Models with Representations from Speech Foundation Modelscited by 1
  5. 2024Sequence-to-sequence models in peer-to-peer learning: A practical applicationcited by 0
  6. 2026SpeechLLM Meets Federated Learning for End-to-End ASR: English and Italian Case Studiescited by 0
  7. 2026Federated Multilingual Speech-LLMs: Architecture and Aggregation Strategy Benchmarkingcited by 0