mlsanityData to 7 Oct 2026

Continual Learning for Monolingual End-to-End Automatic Speech Recognition

arXiv 2112.09427 ↗First posted 17 Dec 2021eess.AS cs.CL cs.LGcited by 8

Not on the map. Why?

Abstract (arXiv)

Adapting Automatic Speech Recognition (ASR) models to new domains results in a deterioration of performance on the original domain(s), a phenomenon called Catastrophic Forgetting (CF). Even monolingual ASR models cannot be extended to new accents, dialects, topics, etc. without suffering from CF, making them unable to be continually enhanced without storing all past data. Fortunately, Continual Learning (CL) methods, which aim to enable continual adaptation while overcoming CF, can be used. In this paper, we implement an extensive number of CL methods for End-to-End ASR and test and compare their ability to extend a monolingual Hybrid CTC-Transformer model across four new tasks. We find that the best performing CL method closes the gap between the fine-tuned model (lower bound) and the model trained jointly on all tasks (upper bound) by more than 40%, while requiring access to only 0.6% of the original data.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 17 of its 30 references that are in the map. First posted on arXiv 17 Dec 2021.

Builds on

17 of 30 references.

References · 17

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  3. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  4. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  5. 2016Learning without Forgettingcited by 2,465
  6. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,973
  7. 2017Gradient Episodic Memory for Continual Learningcited by 1,871
  8. 2017Memory Aware Synapses: Learning what (not) to forgetcited by 1,094
  9. 2018Experience Replay for Continual Learningcited by 1,074
  10. 2018Efficient Lifelong Learning with A-GEMcited by 1,044
  11. 2018ESPnet: End-to-End Speech Processing Toolkitcited by 554
  12. 2018Learning without Memorizingcited by 249
5 more
  1. 2021SynthASR: Unlocking Synthetic Data for Speech Recognitioncited by 34
  2. 2021Towards Lifelong Learning of End-to-end ASRcited by 20
  3. 2020Gradient Episodic Memory with a Soft Constraint for Continual Learningcited by 12
  4. 2020Incremental Learning for End-to-End Automatic Speech Recognitioncited by 10
  5. 2020Batch-level Experience Replay with Review for Continual Learningcited by 10

Built on by

Cited by year

Citing papers per year: 2022: 3; 2023: 2; 2024: 3.

8 papers.

  1. 2022Weight Averaging: A Simple Yet Effective Method to Overcome Catastrophic Forgetting in Automatic Speech Recognitioncited by 10
  2. 2024Towards Rehearsal-Free Multilingual ASR: A LoRA-based Case Study on Whispercited by 9
  3. 2022Continual Learning for On-Device Speech Recognition using Disentangled Conformerscited by 4
  4. 2023Replay to Remember: Continual Layer-Specific Fine-tuning for German Speech Recognitioncited by 4
  5. 2024Continuously Learning New Words in Automatic Speech Recognitioncited by 3
  6. 2024A Parameter-efficient Language Extension Framework for Multilingual ASRcited by 3
  7. 2023Code-Switched Urdu ASR for Noisy Telephonic Environment using Data Centric Approach with Hybrid HMM and CNN-TDNNcited by 2
  8. 2022Damage Control During Domain Adaptation for Transducer Based Automatic Speech Recognitioncited by 1