mlsanityData to 14 Oct 2026

Dual-Pipeline with Low-Rank Adaptation for New Language Integration in Multilingual ASR

arXiv 2406.07842 ↗First posted 12 Jun 2024eess.AS cs.CLcited by 1

Not on the map. Why?

Abstract (arXiv)

This paper addresses challenges in integrating new languages into a pre-trained multilingual automatic speech recognition (mASR) system, particularly in scenarios where training data for existing languages is limited or unavailable. The proposed method employs a dual-pipeline with low-rank adaptation (LoRA). It maintains two data flow pipelines-one for existing languages and another for new languages. The primary pipeline follows the standard flow through the pre-trained parameters of mASR, while the secondary pipeline additionally utilizes language-specific parameters represented by LoRA and a separate output decoder module. Importantly, the proposed approach minimizes the performance degradation of existing languages and enables a language-agnostic operation mode, facilitated by a decoder selection strategy. We validate the effectiveness of the proposed method by extending the pre-trained Whisper model to 19 new languages from the FLEURS dataset

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 20 of its 23 references that are in the map. First posted on arXiv 12 Jun 2024.

Builds on

20 of 23 references.

References · 20

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  4. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  5. 2016Layer Normalizationcited by 5,669
  6. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
  7. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  8. 2018Continual Lifelong Learning with Neural Networks: A Reviewcited by 1,330
  9. 2021Towards a Unified View of Parameter-Efficient Transfer Learningcited by 872
  10. 2015Listen, Attend and Spellcited by 543
  11. 2022FLEURS: Few-shot Learning Evaluation of Universal Representations of Speechcited by 469
  12. 2023Scaling Speech Technology to 1,000+ Languagescited by 411
8 more
  1. 2021Parameter-efficient Multi-task Fine-tuning for Transformers via Shared Hypernetworkscited by 284
  2. 2023Google USM: Scaling Automatic Speech Recognition Beyond 100 Languagescited by 231
  3. 2019Neural Machine Translation with Byte-Level Subwordscited by 115
  4. 2020Massively Multilingual ASR: 50 Languages, 1 Model, 1 Billion Parameterscited by 110
  5. 2021Scaling End-to-End Models for Large-Scale Multilingual ASRcited by 61
  6. 2022Efficient Adapter Transfer of Self-Supervised Speech Models for Automatic Speech Recognitioncited by 32
  7. 2022Exploring Efficient-tuning Methods in Self-supervised Speech Modelscited by 26
  8. 2018Accelerating Neural Transformer via an Average Attention Networkcited by 22

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026A Language-Agnostic Hierarchical LoRA-MoE Architecture for CTC-based Multilingual ASRcited by 1