mlsanityData to 7 Oct 2026

Lightweight Adapter Tuning for Multilingual Speech Translation

arXiv 2106.01463 ↗First posted 2 Jun 2021cs.CLcited by 53

Not on the map. Why?

Abstract (arXiv)

Adapter modules were recently introduced as an efficient alternative to fine-tuning in NLP. Adapter tuning consists in freezing pretrained parameters of a model and injecting lightweight modules between layers, resulting in the addition of only a small number of task-specific trainable parameters. While adapter tuning was investigated for multilingual neural machine translation, this paper proposes a comprehensive analysis of adapters for multilingual speech translation (ST). Starting from different pre-trained models (a multilingual ST trained on parallel data or a multilingual BART (mBART) trained on non-parallel multilingual data), we show that adapters can be used to: (a) efficiently specialize ST to specific language pairs with a low extra cost in terms of parameters, and (b) transfer from an automatic speech recognition (ASR) task and an mBART pre-trained model to a multilingual ST task. Experiments show that adapter tuning offer competitive results to full fine-tuning, while being much more parameter-efficient.

Key facts. Cited by 53 papers in the map (data through 7 Oct 2026). Builds on 25 of its 42 references that are in the map. First posted on arXiv 2 Jun 2021.

Builds on

25 of 42 references.

References · 25

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  4. 2016Layer Normalizationcited by 5,579
  5. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  6. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  7. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,973
  8. 2019SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognitioncited by 1,355
  9. 2020Multilingual Denoising Pre-training for Neural Machine Translationcited by 1,028
  10. 2020AdapterFusion: Non-Destructive Task Composition for Transfer Learningcited by 780
  11. 2017Learning multiple visual domains with residual adapterscited by 553
  12. 2020MAD-X: An Adapter-Based Framework for Multi-Task Cross-Lingual Transfercited by 443
13 more
  1. 2020AdapterHub: A Framework for Adapting Transformerscited by 441
  2. 2020Multilingual Translation with Extensible Multilingual Pretraining and Finetuningcited by 287
  3. 2019Simple, Scalable Adaptation for Neural Machine Translationcited by 239
  4. 2020fairseq S2T: Fast Speech-to-Text Modeling with fairseqcited by 174
  5. 2019Large-Scale Multilingual Speech Recognition with a Streaming End-to-End Modelcited by 92
  6. 2020ESPnet-ST: All-in-One Speech Translation Toolkitcited by 82
  7. 2018Pre-training on high-resource speech recognition improves low-resource speech-to-text translationcited by 70
  8. 2020Dual-decoder Transformer for Joint Automatic Speech Recognition and Multilingual Speech Translationcited by 54
  9. 2020Adapt-and-Adjust: Overcoming the Long-Tail Problem of Multilingual Speech Recognitioncited by 34
  10. 2020Recipes for Adapting Pre-trained Monolingual and Multilingual Models to Machine Translationcited by 28
  11. 2021End-to-End Speech Translation with Pre-trained Models and Adapters: UPC at IWSLT 2021cited by 16
  12. 2020Enabling Zero-shot Multilingual Spoken Language Translation with Language-Specific Encoders and Decoderscited by 10
  13. 2020Adaptable Multi-Domain Language Model for Transformer ASRcited by 3

Built on by

Cited by year

Citing papers per year: 2021: 4; 2022: 18; 2023: 21; 2024: 8; 2025: 2.

53 papers.

Sort:

  1. 2021SpeechT5: Unified-Modal Encoder-Decoder Pre-Training for Spoken Language Processingcited by 177
  2. 2022Lifting the Curse of Multilinguality by Pre-training Modular Transformerscited by 145
  3. 2023MotionGPT: Finetuned LLMs Are General-Purpose Motion Generatorscited by 145
  4. 2022Pretrained Language Models for Text Generation: A Surveycited by 121
  5. 2023Modular Deep Learningcited by 82
  6. 2021xGQA: Cross-Lingual Visual Question Answeringcited by 72
  7. 2022Cross-modal Contrastive Learning for Speech Translationcited by 61
  8. 2024MotionGPT-2: A General-Purpose Motion-Language Model for Motion Generation and Understandingcited by 58
  9. 2023AlignAtt: Using Attention-based Audio-Translation Alignments as a Guide for Simultaneous Speech Translationcited by 35
  10. 2023Empowering Cross-lingual Abilities of Instruction-tuned Large Language Models by Translation-following demonstrationscited by 34
  11. 2022Efficient Adapter Transfer of Self-Supervised Speech Models for Automatic Speech Recognitioncited by 31
  12. 2021Learning When to Translate for Streaming Speechcited by 30
38 more
  1. 2022Revisiting End-to-End Speech-to-Text Translation From Scratchcited by 26
  2. 2023Pre-training for Speech Translation: CTC Meets Optimal Transportcited by 25
  3. 2023mmT5: Modular Multilingual Pre-Training Solves Source Language Hallucinationscited by 24
  4. 2022Under the Morphosyntactic Lens: A Multifaceted Evaluation of Gender Bias in Speech Translationcited by 23
  5. 2023Dynamics Generalisation in Reinforcement Learning via Adaptive Context-Aware Policiescited by 18
  6. 2021Regularizing End-to-End Speech Translation with Triangular Decomposition Agreementcited by 16
  7. 2023Understanding and Bridging the Modality Gap for Speech Translationcited by 15
  8. 2023Recent Advances in Direct Speech-to-text Translationcited by 15
  9. 2023Multilingual DistilWhisper: Efficient Distillation of Multi-task Speech Models via Language-Specific Expertscited by 15
  10. 2022Non-Parametric Domain Adaptation for End-to-End Speech Translationcited by 12
  11. 2022Residual Adapters for Few-Shot Text-to-Speech Speaker Adaptationcited by 10
  12. 2022Adaptive multilingual speech recognition with pretrained modelscited by 9
  13. 2023Back Translation for Speech-to-text Translation Without Transcriptscited by 9
  14. 2023ComSL: A Composite Speech-Language Model for End-to-End Speech-to-Text Translationcited by 9
  15. 2023Master-ASR: Achieving Multilingual Scalability and Low-Resource Adaptation in ASR with Modular Learningcited by 9
  16. 2023End-to-End Speech-to-Text Translation: A Surveycited by 9
  17. 2022Genre-conditioned Acoustic Models for Automatic Lyrics Transcription of Polyphonic Musiccited by 7
  18. 2022Losses Can Be Blessings: Routing Self-Supervised Speech Representations Towards Efficient Multilingual and Multitask Speech Processingcited by 7
  19. 2025In-Context Learning Boosts Speech Recognition via Human-like Adaptation to Speakers and Language Varietiescited by 7
  20. 2022Discrete Cross-Modal Alignment Enables Zero-Shot Speech Translationcited by 6
  21. 2023Efficient Domain Adaptation of Sentence Embeddings Using Adapterscited by 6
  22. 2023An Empirical Study of Consistency Regularization for End-to-End Speech-to-Text Translationcited by 6
  23. 2025Low-Rank and Sparse Model Merging for Multi-Lingual Speech Recognition and Translationcited by 6
  24. 2023E2TIMT: Efficient and Effective Modal Adapter for Text Image Machine Translationcited by 5
  25. 2024The Impact of LoRA Adapters on LLMs for Clinical Text Classification Under Computational and Data Constraintscited by 4
  26. 2022Introducing Semantics into Speech Encoderscited by 3
  27. 2023Prompting and Adapter Tuning for Self-supervised Encoder-Decoder Speech Modelcited by 3
  28. 2023Towards a Deep Understanding of Multilingual End-to-End Speech Translationcited by 3
  29. 2024ELP-Adapters: Parameter Efficient Adapter Tuning for Various Speech Processing Taskscited by 3
  30. 2022Cross-Lingual Knowledge Transfer for Clinical Phenotypingcited by 2
  31. 2022Structural Bias for Aspect Sentiment Triplet Extractioncited by 2
  32. 2022Data-Efficient Cross-Lingual Transfer with Language-Specific Subnetworkscited by 2
  33. 2023System-Initiated Transitions from Chit-Chat to Task-Oriented Dialogues with Transition Info Extractor and Transition Sentence Generatorcited by 2
  34. 2024Efficient Compression of Multitask Multilingual Speech Modelscited by 2
  35. 2022Efficient Speech Translation with Pre-trained Modelscited by 1
  36. 2023Out-of-Distribution Generalization in Text Classification: Past, Present, and Futurecited by 1
  37. 2024Hypernetworks for Personalizing ASR to Atypical Speechcited by 1
  38. 2024Leveraging Parameter-Efficient Transfer Learning for Multi-Lingual Text-to-Speech Adaptationcited by 1

50 most cited of 53 All 53 (JSON)