Data to 7 Oct 2026

Simple, Scalable Adaptation for Neural Machine Translation

arXiv 1909.08478 ↗First posted 18 Sep 2019cs.CL cs.LGcited by 239

Not on the map. Why?

Abstract (arXiv)

Fine-tuning pre-trained Neural Machine Translation (NMT) models is the dominant approach for adapting to new languages and domains. However, fine-tuning requires adapting and maintaining a separate model for each target task. We propose a simple yet efficient approach for adaptation in NMT. Our proposed approach consists of injecting tiny task specific adapter layers into a pre-trained model. These lightweight adapters, with just a small fraction of the original model size, adapt the model to multiple individual tasks simultaneously. We evaluate our approach on two tasks: (i) Domain Adaptation and (ii) Massively Multilingual NMT. Experiments on domain adaptation demonstrate that our proposed approach is on par with full fine-tuning on various domains, dataset sizes and model capacities. On a massively multilingual dataset of 103 languages, our adaptation approach bridges the gap between individual bilingual models and one massively multilingual model for most language pairs, paving the way towards universal machine translation.

Key facts. Cited by 239 papers in the map (data through 7 Oct 2026). First posted on arXiv 18 Sep 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 50; 2022: 57; 2023: 65; 2024: 49; 2025: 12; 2026: 6.

239 papers.

Sort:

By area:

2022Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learningcited by 886
2022AdaptFormer: Adapting Vision Transformers for Scalable Visual Recognitioncited by 714
2021Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Surveycited by 457
2021BASE Layers: Simplifying Training of Large, Sparse Modelscited by 315
2021Parameter-efficient Multi-task Fine-tuning for Transformers via Shared Hypernetworkscited by 278
2023Parameter-Efficient Fine-Tuning Methods for Pretrained Language Models: A Critical Review and Assessmentcited by 227
2021Training Neural Networks with Fixed Sparse Maskscited by 212
2024When Scaling Meets LLM Finetuning: The Effect of Data, Model and Finetuning Methodcited by 201
2022ST-Adapter: Parameter-Efficient Image-to-Video Transfer Learningcited by 192
2023Pushing Mixture of Experts to the Limit: Extremely Parameter Efficient MoE for Instruction Tuningcited by 148
2021On the Effectiveness of Adapter-based Tuning for Pretrained Language Model Adaptationcited by 147
2022Lifting the Curse of Multilinguality by Pre-training Modular Transformerscited by 145

38 more

2021Composable Sparse Fine-Tuning for Cross-Lingual Transfercited by 144
2023Domain Specialization as the Key to Make Large Language Models Disruptive: A Comprehensive Surveycited by 134
2021Contrastive Learning for Many-to-many Multilingual Neural Machine Translationcited by 108
2021Beyond Distillation: Task-level Mixture-of-Experts for Efficient Inferencecited by 96
2023AdapterSoup: Weight Averaging to Improve Generalization of Pretrained Language Modelscited by 84
2023Modular Deep Learningcited by 82
2024Chain of LoRA: Efficient Fine-tuning of Language Models via Residual Learningcited by 80
2022Efficient Methods for Natural Language Processing: A Surveycited by 78
2021Cross-Attention is All You Need: Adapting Pretrained Transformers for Machine Translationcited by 74
2021xGQA: Cross-Lingual Visual Question Answeringcited by 72
2024LoRA-Pro: Are Low-Rank Adapters Properly Optimized?cited by 70
2021Improving Speech Translation by Understanding and Learning from the Auxiliary Text Translation Taskcited by 54
2021Lightweight Adapter Tuning for Multilingual Speech Translationcited by 53
2021A Survey on Green Deep Learningcited by 52
2022Singular Value Fine-tuning: Few-shot Segmentation requires Few-parameters Fine-tuningcited by 50
2021Learning Language Specific Sub-network for Multilingual Machine Translationcited by 48
2021Domain Adaptation and Multi-Domain Adaptation for Neural Machine Translation: A Surveycited by 46
2023Mitigating Hallucinations and Off-target Machine Translation with Source-Contrastive and Language-Contrastive Decodingcited by 42
2023Gradient-based Parameter Selection for Efficient Fine-Tuningcited by 40
2021Counter-Interference Adapter for Multilingual Machine Translationcited by 39
2023Incorporating Structured Representations into Pretrained Vision & Language Models Using Scene Graphscited by 39
2024GOAL: A Generalist Combinatorial Optimization Agent Learnercited by 39
2024Leveraging Large Language Models for NLG Evaluation: Advances and Challengescited by 37
2021MSP: Multi-Stage Prompting for Making Pre-trained Language Models Better Translatorscited by 35
2021Lexicon Enhanced Chinese Sequence Labeling Using BERT Adaptercited by 34
2021Exploiting Adapters for Cross-lingual Low-resource Speech Recognitioncited by 33
2023eP-ALM: Efficient Perceptual Augmentation of Language Modelscited by 33
2021Efficient Hierarchical Domain Adaptation for Pretrained Language Modelscited by 32
2024Mixture of In-Context Prompters for Tabular PFNscited by 32
2022Efficient Adapter Transfer of Self-Supervised Speech Models for Automatic Speech Recognitioncited by 31
2021Residual Adapters for Parameter-Efficient ASR Adaptation to Atypical and Accented Speechcited by 30
2023Task-Optimized Adapters for an End-to-End Task-Oriented Dialogue Systemcited by 30
2023MISSRec: Pre-training and Transferring Multi-modal Interest-aware Sequence Representation for Recommendationcited by 30
2022AdapterBias: Parameter-efficient Token-dependent Representation Shift for Adapters in NLP Taskscited by 29
2023Learning to Modulate pre-trained Models in RLcited by 28
2023Language and Task Arithmetic with Parameter-Efficient Layers for Zero-Shot Summarizationcited by 28
2024Gradient Routing: Masking Gradients to Localize Computation in Neural Networkscited by 28
2021Attribute Alignment: Controlling Text Generation from Pre-trained Language Modelscited by 27

50 most cited of 239 All 239 (JSON)