Data to 7 Oct 2026

Improving Massively Multilingual Neural Machine Translation and Zero-Shot Translation

arXiv 2004.11867 ↗First posted 24 Apr 2020cs.CLcited by 277

Not on the map. Why?

Abstract (arXiv)

Massively multilingual models for neural machine translation (NMT) are theoretically attractive, but often underperform bilingual models and deliver poor zero-shot translations. In this paper, we explore ways to improve them. We argue that multilingual NMT requires stronger modeling capacity to support language pairs with varying typological characteristics, and overcome this bottleneck via language-specific components and deepening NMT architectures. We identify the off-target translation issue (i.e. translating into a wrong target language) as the major source of the inferior zero-shot performance, and propose random online backtranslation to enforce the translation of unseen training language pairs. Experiments on OPUS-100 (a novel multilingual dataset with 100 languages) show that our approach substantially narrows the performance gap with bilingual models in both one-to-many and many-to-many settings, and improves zero-shot performance by ~10 BLEU, approaching conventional pivot-based methods.

Key facts. Cited by 277 papers in the map (data through 7 Oct 2026). First posted on arXiv 24 Apr 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 55; 2022: 51; 2023: 62; 2024: 56; 2025: 27; 2026: 26.

277 papers.

Sort:

By area:

2021The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translationcited by 556
2023Prompting Large Language Model for Machine Translation: A Case Studycited by 247
2024Tower: An Open Multilingual Large Language Model for Translation-Related Taskscited by 205
2023MADLAD-400: A Multilingual And Document-Level Large Audited Datasetcited by 180
2022DeepNet: Scaling Transformers to 1,000 Layerscited by 176
2023The BigScience ROOTS Corpus: A 1.6TB Composite Multilingual Datasetcited by 148
2023Hallucinations in Large Multilingual Translation Modelscited by 126
2023Accelerating Transformer Inference for Translation via Parallel Decodingcited by 123
2021Contrastive Learning for Many-to-many Multilingual Neural Machine Translationcited by 108
2021Neural Machine Translation for Low-Resource Languages: A Surveycited by 108
2022State-of-the-art generalisation research in NLP: A taxonomy and reviewcited by 97
2021Beyond Distillation: Task-level Mixture-of-Experts for Efficient Inferencecited by 96

38 more

2021Survey of Low-Resource Machine Translationcited by 92
2022Building Machine Translation Systems for the Next Thousand Languagescited by 69
2024Large Language Model Agent for Hyper-Parameter Optimizationcited by 62
2022Cross-modal Contrastive Learning for Speech Translationcited by 61
2023BigTranslate: Augmenting Large Language Models with Multilingual Translation Capability over 100 Languagescited by 60
2024Adapting Large Language Models for Document-Level Machine Translationcited by 59
2021Facebook AI WMT21 News Translation Task Submissioncited by 57
2023Eliciting the Translation Ability of Large Language Models via Multilingual Finetuning with Translation Instructionscited by 55
2021Improving Speech Translation by Understanding and Learning from the Auxiliary Text Translation Taskcited by 54
2024LLMs Beyond English: Scaling the Multilingual Capability of LLMs with Cross-Lingual Feedbackcited by 51
2021Learning Shared Semantic Space for Speech-to-Text Translationcited by 49
2023An Open Dataset and Model for Language Identificationcited by 49
2024Multilingual Large Language Model: A Survey of Resources, Taxonomy and Frontierscited by 49
2021Learning Language Specific Sub-network for Multilingual Machine Translationcited by 48
2021End-to-end Speech Translation via Cross-modal Progressive Trainingcited by 47
2021Domain Adaptation and Multi-Domain Adaptation for Neural Machine Translation: A Surveycited by 46
2021MURAL: Multimodal, Multitask Retrieval Across Languagescited by 44
2024Fine-tuning Large Language Models for Domain-specific Machine Translationcited by 43
2024Segment Any Text: A Universal Approach for Robust, Efficient and Adaptable Sentence Segmentationcited by 41
2021Counter-Interference Adapter for Multilingual Machine Translationcited by 39
2024Language Ranker: A Metric for Quantifying LLM Performance Across High and Low-Resource Languagescited by 38
2023GlotLID: Language Identification for Low-Resource Languagescited by 37
2022Memory-efficient NLLB-200: Language-specific Expert Pruning of a Massively Multilingual Machine Translation Modelcited by 36
2024Enhancing Multilingual Capabilities of Large Language Models through Self-Distillation from Resource-Rich Languagescited by 36
2024X-ALMA: Plug & Play Modules and Adaptive Rejection for Quality Translation at Scalecited by 36
2023SLTUNET: A Simple Unified Model for Sign Language Translationcited by 35
2021Multilingual Machine Translation Systems from Microsoft for WMT21 Shared Taskcited by 34
2023Improving In-context Learning of Multilingual Generative Language Models with Cross-lingual Alignmentcited by 33
2021Many-to-English Machine Translation Tools, Data, and Pretrained Modelscited by 31
2021Improving Multilingual Translation by Representation and Gradient Regularizationcited by 31
2023ResiDual: Transformer with Dual Residual Connectionscited by 30
2024Tag-LLM: Repurposing General-Purpose LLMs for Specialized Domainscited by 30
2021Language Tags Matter for Zero-Shot Neural Machine Translationcited by 29
2021From Machine Translation to Code-Switching: Generating High-Quality Code-Switched Textcited by 29
2022Using natural language prompts for machine translationcited by 27
2022Examining Scaling and Transfer of Language Model Architectures for Machine Translationcited by 26
2022Insights into Pre-training via Simpler Synthetic Taskscited by 25
2024MEXA: Multilingual Evaluation of English-Centric LLMs via Cross-Lingual Alignmentcited by 25

50 most cited of 277 All 277 (JSON)