mlsanityData to 7 Oct 2026

CL-MASR: A Continual Learning Benchmark for Multilingual ASR

arXiv 2310.16931 ↗First posted 25 Oct 2023cs.CL cs.AIcited by 16

Not on the map. Why?

Abstract (arXiv)

Modern multilingual automatic speech recognition (ASR) systems like Whisper have made it possible to transcribe audio in multiple languages with a single model. However, current state-of-the-art ASR models are typically evaluated on individual languages or in a multi-task setting, overlooking the challenge of continually learning new languages. There is insufficient research on how to add new languages without losing valuable information from previous data. Furthermore, existing continual learning benchmarks focus mostly on vision and language tasks, leaving continual learning for multilingual ASR largely unexplored. To bridge this gap, we propose CL-MASR, a benchmark designed for studying multilingual ASR in a continual learning setting. CL-MASR provides a diverse set of continual learning methods implemented on top of large-scale pretrained ASR models, along with common metrics to assess the effectiveness of learning new languages while addressing the issue of catastrophic forgetting. To the best of our knowledge, CL-MASR is the first continual learning benchmark for the multilingual ASR task. The code is available at https://github.com/speechbrain/benchmarks.

Key facts. Cited by 16 papers in the map (data through 7 Oct 2026). Builds on 64 of its 84 references that are in the map. First posted on arXiv 25 Oct 2023.

Builds on

64 of 84 references.

References · 64

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  3. 2020Scaling Laws for Neural Language Modelscited by 6,411
  4. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  5. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
  6. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  7. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  8. 2016Learning without Forgettingcited by 2,465
  9. 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,313
  10. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,301
  11. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,973
  12. 2017Gradient Episodic Memory for Continual Learningcited by 1,871
38 more
  1. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,838
  2. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,609
  3. 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,358
  4. 2016Progressive Neural Networkscited by 1,332
  5. 2017Continual Learning with Deep Generative Replaycited by 1,152
  6. 2017Memory Aware Synapses: Learning what (not) to forgetcited by 1,094
  7. 2018Experience Replay for Continual Learningcited by 1,074
  8. 2018Efficient Lifelong Learning with A-GEMcited by 1,044
  9. 2021Learning to Prompt for Continual Learningcited by 938
  10. 2020Dark Experience for General Continual Learning: a Strong, Simple Baselinecited by 895
  11. 2017PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruningcited by 882
  12. 2023A Comprehensive Survey of Continual Learning: Theory, Method and Applicationcited by 860
  13. 2018Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigencecited by 734
  14. 2013An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networkscited by 711
  15. 2023Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizescited by 710
  16. 2021SUPERB: Speech processing Universal PERformance Benchmarkcited by 652
  17. 2021The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translationcited by 556
  18. 2017Learning multiple visual domains with residual adapterscited by 553
  19. 2020MLS: A Large-Scale Multilingual Dataset for Speech Researchcited by 467
  20. 2022FLEURS: Few-shot Learning Evaluation of Universal Representations of Speechcited by 458
  21. 2021VoxPopuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and Interpretationcited by 455
  22. 2020Class-incremental learning: survey and performance evaluation on image classificationcited by 446
  23. 2021SpeechBrain: A General-Purpose Speech Toolkitcited by 404
  24. 2023Scaling Speech Technology to 1,000+ Languagescited by 401
  25. 2016Expert Gate: Lifelong Learning with a Network of Expertscited by 369
  26. 2017CORe50: a New Dataset and Benchmark for Continuous Object Recognitioncited by 287
  27. 2021Online Continual Learning in Image Classification: An Empirical Surveycited by 242
  28. 2018The Natural Language Decathlon: Multitask Learning as Question Answeringcited by 235
  29. 2023Google USM: Scaling Automatic Speech Recognition Beyond 100 Languagescited by 228
  30. 2022Self-supervised Learning with Random-projection Quantizer for Speech Recognitioncited by 158
  31. 2019Continual Unsupervised Representation Learningcited by 147
  32. 2021Avalanche: an End-to-End Library for Continual Learningcited by 124
  33. 2021Representational Continuity for Unsupervised Continual Learningcited by 95
  34. 2022The CLEAR Benchmark: Continual LEArning on Real-World Imagerycited by 92
  35. 2022Continual Learning of Natural Language Processing Tasks: A Surveycited by 83
  36. 2023ML-SUPERB: Multilingual Speech Universal PERformance Benchmarkcited by 70
  37. 2022CLiMB: A Continual Learning Benchmark for Vision-and-Language Taskscited by 68
  38. 2020MixKD: Towards Efficient Distillation of Large-scale Language Modelscited by 65

50 most cited of 64 All 64 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 6; 2025: 3; 2026: 7.

16 papers.

Sort:

By area:

  1. 2024Open-Source Conversational AI with SpeechBrain 1.0cited by 55
  2. 2024LoRA-Whisper: Parameter-Efficient and Extensible Multilingual ASRcited by 35
  3. 2024ML-SUPERB 2.0: Benchmarking Multilingual Speech Models Across Modeling Constraints, Languages, and Datasetscited by 26
  4. 2024Continual Learning Optimizations for Auto-regressive Decoder of Multilingual ASR systemscited by 7
  5. 2024A Parameter-efficient Language Extension Framework for Multilingual ASRcited by 3
  6. 2025NIRANTAR: Continual Learning with New Languages and Domains on Real-world Speech Datacited by 2
  7. 2024Double Mixture: Towards Continual Event Detection from Speechcited by 1
  8. 2025Sample Compression for Self Certified Continual Learningcited by 1
  9. 2025A Study on Regularization-Based Continual Learning Methods for Indic ASRcited by 1
  10. 2026Adapting Where It Matters: Depth-Aware Adaptation for Efficient Multilingual Speech Recognition in Low-Resource Languagescited by 1
  11. 2026BBPE16: UTF-16-based byte-level byte-pair encoding for improved multilingual speech recognitioncited by 0
  12. 2026Continual Adaptation for Pacific Indigenous Speech Recognitioncited by 0
4 more
  1. 2026Learning to Hear Hesitation: Continual Learning for Disfluency-Aware ASRcited by 0
  2. 2026MoDiCoL: A Modular Diagnostic Continual Learning Dataset for Robust Speech Recognitioncited by 0
  3. 2026MambAdapter: Lightweight Mamba-Based Adapters for Parameter-Efficient Transfer Learning in Speech and Audiocited by 0
  4. 2026Unified Gradient Projection: Language-Balanced Continual Learning for Multilingual Low-Resource ASRcited by 0