mlsanityData to 7 Oct 2026

Multilingual Machine Translation Systems from Microsoft for WMT21 Shared Task

arXiv 2111.02086 ↗First posted 3 Nov 2021cs.CLcited by 34

Not on the map. Why?

Abstract (arXiv)

This report describes Microsoft's machine translation systems for the WMT21 shared task on large-scale multilingual machine translation. We participated in all three evaluation tracks including Large Track and two Small Tracks where the former one is unconstrained and the latter two are fully constrained. Our model submissions to the shared task were initialized with DeltaLMfootnoteurlhttps://aka.ms/deltalm, a generic pre-trained multilingual encoder-decoder model, and fine-tuned correspondingly with the vast collected parallel data and allowed data sources according to track settings, together with applying progressive learning and iterative back-translation approaches to further improve the performance. Our final submissions ranked first on three tracks in terms of the automatic evaluation metric.

Key facts. Cited by 34 papers in the map (data through 7 Oct 2026). Builds on 22 of its 31 references that are in the map. First posted on arXiv 3 Nov 2021.

Builds on

22 of 31 references.

References · 22

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,973
  3. 2020Multilingual Denoising Pre-training for Neural Machine Translationcited by 1,028
  4. 2020Beyond English-Centric Multilingual Machine Translationcited by 559
  5. 2021The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translationcited by 556
  6. 2016Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translationcited by 467
  7. 2020Improving Massively Multilingual Neural Machine Translation and Zero-Shot Translationcited by 277
  8. 2019Massively Multilingual Neural Machine Translation in the Wild: Findings and Challengescited by 253
  9. 2019Massively Multilingual Neural Machine Translationcited by 246
  10. 2019CCMatrix: Mining Billions of High-Quality Parallel Sentences on the WEBcited by 177
  11. 2019CCAligned: A Massive Collection of Cross-Lingual Web-Document Pairscited by 136
  12. 2021Contrastive Learning for Many-to-many Multilingual Neural Machine Translationcited by 108
10 more
  1. 2021Learning with Feature-Dependent Label Noise: A Progressive Approachcited by 102
  2. 2019Multilingual Neural Machine Translation with Knowledge Distillationcited by 95
  3. 2020Balancing Training for Multilingual Neural Machine Translationcited by 71
  4. 2021DeltaLM: Encoder-Decoder Pre-training for Language Generation and Translation by Augmenting Pretrained Multilingual Encoderscited by 50
  5. 2021Learning Language Specific Sub-network for Multilingual Machine Translationcited by 48
  6. 2020Multi-task Learning for Multilingual Neural Machine Translationcited by 46
  7. 2019Pivot-based Transfer Learning for Neural Machine Translation between Non-English Languagescited by 36
  8. 2020Shallow-to-Deep Training for Neural Machine Translationcited by 36
  9. 2020Dynamic Data Selection and Weighting for Iterative Back-Translationcited by 25
  10. 2021Adaptive Sparse Transformer for Multilingual Translationcited by 8

Built on by

Cited by year

Citing papers per year: 2022: 12; 2023: 11; 2024: 10; 2025: 1.

34 papers.

Sort:

By area:

  1. 2023Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyondcited by 409
  2. 2024OpenCoder: The Open Cookbook for Top-Tier Code Large Language Modelscited by 127
  3. 2023SCM: Enhancing Large Language Model with Self-Controlled Memory Frameworkcited by 98
  4. 2022A Few Thousand Translations Go a Long Way! Leveraging Pre-trained Models for African News Translationcited by 97
  5. 2024C-ICL: Contrastive In-context Learning for Information Extractioncited by 22
  6. 2022GanLM: Encoder-Decoder Pre-training with an Auxiliary Discriminatorcited by 20
  7. 2024Lemur: Log Parsing with Entropy Sampling and Chain-of-Thought Mergingcited by 16
  8. 2025P2P: Automated Paper-to-Poster Generation and Fine-Grained Benchmarkcited by 15
  9. 2023mCL-NER: Cross-Lingual Named Entity Recognition via Multi-view Contrastive Learningcited by 14
  10. 2022Tencent's Multilingual Machine Translation System for WMT22 Large-Scale African Languagescited by 13
  11. 2023Learning Language-Specific Layers for Multilingual Machine Translationcited by 13
  12. 2023FinPT: Financial Risk Prediction with Profile Tuning on Pretrained Foundation Modelscited by 12
22 more
  1. 2022GTrans: Grouping and Fusing Transformer Layers for Neural Machine Translationcited by 10
  2. 2023Towards Higher Pareto Frontier in Multilingual Machine Translationcited by 9
  3. 2023M2C: Towards Automatic Multimodal Manga Complementcited by 7
  4. 2023Adaptive Neural Ranking Framework: Toward Maximized Business Goal for Cascade Ranking Systemscited by 6
  5. 2024Leveraging Large Language Models for Enhanced NLP Task Performance through Knowledge Distillation and Optimized Training Strategiescited by 6
  6. 2024New Intent Discovery with Attracting and Dispersing Prototypecited by 6
  7. 2024m3P: Towards Multimodal Multilingual Translation with Multimodal Promptcited by 5
  8. 2022Building Multilingual Machine Translation Systems That Serve Arbitrary X-Y Translationscited by 4
  9. 2022Learning an Artificial Language for Knowledge-Sharing in Multilingual Translationcited by 4
  10. 2023HanoiT: Enhancing Context-aware Translation via Selective Contextcited by 4
  11. 2022The YiTrans End-to-End Speech Translation System for IWSLT 2022 Offline Shared Taskcited by 3
  12. 2022Tackling Low-Resourced Sign Language Translation: UPC at WMT-SLT 22cited by 3
  13. 2023Multilingual Entity and Relation Extraction from Unified to Language-specific Trainingcited by 3
  14. 2022Advancing Multilingual Pre-training: TRIP Triangular Document-level Pre-training for Multilingual Language Modelscited by 2
  15. 2024RoNID: New Intent Discovery with Generated-Reliable Labels and Cluster-friendly Representationscited by 2
  16. 2024Towards Real-world Scenario: Imbalanced New Intent Discoverycited by 2
  17. 2022Fast Vocabulary Projection Method via Clustering for Multilingual Machine Translation on GPUcited by 1
  18. 2024Low-Resource Machine Translation through the Lens of Personalized Federated Learningcited by 1
  19. 2022Revamping Multilingual Agreement Bidirectionally via Switched Back-translation for Multilingual Neural Machine Translationcited by 0
  20. 2022The VolcTrans System for WMT22 Multilingual Machine Translation Taskcited by 0
  21. 2023One-stop Training of Multiple Capacity Modelscited by 0
  22. 2024SPRING Lab IITM's submission to Low Resource Indic Language Translation Shared Taskcited by 0