mlsanityData to 14 Oct 2026

Language-Aware Prompt Tuning for Parameter-Efficient Seamless Language Expansion in Multilingual ASR

arXiv 2506.21577 ↗First posted 16 Jun 2025cs.CL cs.AI cs.SDcited by 2

Not on the map. Why?

Abstract (arXiv)

Recent advancements in multilingual automatic speech recognition (ASR) have been driven by large-scale end-to-end models like Whisper. However, challenges such as language interference and expanding to unseen languages (language expansion) without degrading performance persist. This paper addresses these with three contributions: 1) Entire Soft Prompt Tuning (Entire SPT), which applies soft prompts to both the encoder and decoder, enhancing feature extraction and decoding; 2) Language-Aware Prompt Tuning (LAPT), which leverages cross-lingual similarities to encode shared and language-specific features using lightweight prompt matrices; 3) SPT-Whisper, a toolkit that integrates SPT into Whisper and enables efficient continual learning. Experiments across three languages from FLEURS demonstrate that Entire SPT and LAPT outperform Decoder SPT by 5.0% and 16.0% in language expansion tasks, respectively, providing an efficient solution for dynamic, multilingual ASR models with minimal computational overhead.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 29 of its 33 references that are in the map. First posted on arXiv 16 Jun 2025.

Builds on

29 of 33 references.

References · 29

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  2. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  3. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,795
  4. 2018Efficient Lifelong Learning with A-GEMcited by 1,058
  5. 2021P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Taskscited by 689
  6. 2015Listen, Attend and Spellcited by 543
  7. 2023SpeechGPT: Empowering Large Language Models with Intrinsic Cross-Modal Conversational Abilitiescited by 529
  8. 2023SALMONN: Towards Generic Hearing Abilities for Large Language Modelscited by 506
  9. 2023Scaling Speech Technology to 1,000+ Languagescited by 411
  10. 2016Joint CTC-Attention based End-to-End Speech Recognition using Multi-task Learningcited by 257
  11. 2023Google USM: Scaling Automatic Speech Recognition Beyond 100 Languagescited by 231
  12. 2024LLaMA-Omni: Seamless Speech Interaction with Large Language Modelscited by 173
17 more
  1. 2021Recent Advances in End-to-End Automatic Speech Recognitioncited by 140
  2. 2023Residual Prompt Tuning: Improving Prompt Tuning with Residual Reparameterizationcited by 64
  3. 2024LoRA-Whisper: Parameter-Efficient and Extensible Multilingual ASRcited by 36
  4. 2023Language-Routing Mixture of Experts for Multilingual and Code-Switching Speech Recognitioncited by 26
  5. 2023Adaptation of Whisper models to child speech recognitioncited by 26
  6. 2023N-Shot Benchmarking of Whisper on Diverse Arabic Speech Recognitioncited by 20
  7. 2024Learn and Don't Forget: Adding a New Language to ASR Foundation Modelscited by 16
  8. 2023Extending Whisper with prompt tuning to target-speaker ASRcited by 14
  9. 2022Language Adaptive Cross-lingual Speech Representation Learning with Sparse Sharing Sub-networkscited by 13
  10. 2023Online Continual Learning in Keyword Spotting for Low-Resource Devices via Pooling High-Order Temporal Statisticscited by 9
  11. 2024Towards Rehearsal-Free Multilingual ASR: A LoRA-based Case Study on Whispercited by 9
  12. 2023Adapting OpenAI's Whisper for Speech Recognition on Code-Switch Mandarin-English SEAME and ASRU2019 Datasetscited by 8
  13. 2023Rehearsal-Free Online Continual Learning for Automatic Speech Recognitioncited by 7
  14. 2022An Investigation of the Combination of Rehearsal and Knowledge Distillation in Continual Learning for Spoken Language Understandingcited by 5
  15. 2023Building High-accuracy Multilingual ASR with Gated Language Experts and Curriculum Trainingcited by 4
  16. 2023Dynamic ASR Pathways: An Adaptive Masking Approach Towards Efficient Pruning of A Multilingual ASR Modelcited by 3
  17. 2024LoPT: Low-Rank Prompt Tuning for Parameter Efficient Language Modelscited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026A Language-Agnostic Hierarchical LoRA-MoE Architecture for CTC-based Multilingual ASRcited by 1
  2. 2026Adapting Where It Matters: Depth-Aware Adaptation for Efficient Multilingual Speech Recognition in Low-Resource Languagescited by 1