mlsanityData to 7 Oct 2026

Federated Representation Learning for Automatic Speech Recognition

arXiv 2308.02013 ↗First posted 3 Aug 2023cs.SD cs.CL cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Federated Learning (FL) is a privacy-preserving paradigm, allowing edge devices to learn collaboratively without sharing data. Edge devices like Alexa and Siri are prospective sources of unlabeled audio data that can be tapped to learn robust audio representations. In this work, we bring Self-supervised Learning (SSL) and FL together to learn representations for Automatic Speech Recognition respecting data privacy constraints. We use the speaker and chapter information in the unlabeled speech dataset, Libri-Light, to simulate non-IID speaker-siloed data distributions and pre-train an LSTM encoder with the Contrastive Predictive Coding framework with FedSGD. We show that the pre-trained ASR encoder in FL performs as well as a centrally pre-trained model and produces an improvement of 12-15% (WER) compared to no pre-training. We further adapt the federated pre-trained models to a new language, French, and show a 20% (WER) improvement over no pre-training.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 25 of its 30 references that are in the map. First posted on arXiv 3 Aug 2023.

Builds on

25 of 30 references.

References · 25

  1. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  3. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  4. 2019Advances and Open Problems in Federated Learningcited by 3,021
  5. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,301
  6. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,973
  7. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,609
  8. 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,358
  9. 2021XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scalecited by 589
  10. 2020Unsupervised Cross-lingual Representation Learning for Speech Recognitioncited by 496
  11. 2020MLS: A Large-Scale Multilingual Dataset for Speech Researchcited by 467
  12. 2019Libri-Light: A Benchmark for ASR with Limited or No Supervisioncited by 462
13 more
  1. 2022Self-Supervised Speech Representation Learning: A Reviewcited by 258
  2. 2018Federated Learning for Keyword Spottingcited by 95
  3. 2020Training Speech Recognition Models with Federated Learning: A Quality/Cost Frameworkcited by 46
  4. 2021End-to-End Speech Recognition from Federated Acoustic Modelscited by 30
  5. 2020Improving on-device speaker verification using federated learning with privacycited by 29
  6. 2022FedAudio: A Federated Learning Benchmark for Audio Taskscited by 15
  7. 2021Federated Learning in ASR: Not as Easy as You Thinkcited by 14
  8. 2021Magic dust for cross-lingual adaptation of monolingual wav2vec-2.0cited by 12
  9. 2022Semi-FedSER: Semi-supervised Learning for Speech Emotion Recognition On Federated Learning using Multiview Pseudo-Labelingcited by 12
  10. 2022Federated Self-supervised Speech Representations: Are We There Yet?cited by 10
  11. 2022Federated Self-Supervised Learning for Acoustic Event Classificationcited by 5
  12. 2023Federated Self-Learning with Weak Supervision for Speech Recognitioncited by 5
  13. 2022ILASR: Privacy-Preserving Incremental Learning for Automatic Speech Recognition at Production Scalecited by 4

Built on by

Not cited yet.