mlsanityData to 7 Oct 2026

Facebook AI WMT21 News Translation Task Submission

arXiv 2108.03265 ↗First posted 6 Aug 2021cs.CLcited by 57

Not on the map. Why?

Abstract (arXiv)

We describe Facebook's multilingual model submission to the WMT2021 shared task on news translation. We participate in 14 language directions: English to and from Czech, German, Hausa, Icelandic, Japanese, Russian, and Chinese. To develop systems covering all these directions, we focus on multilingual models. We utilize data from all available sources --- WMT, large-scale data mining, and in-domain backtranslation --- to create high quality bilingual and multilingual baselines. Subsequently, we investigate strategies for scaling multilingual model size, such that one system has sufficient capacity for high quality representations of all eight languages. Our final submission is an ensemble of dense and sparse Mixture-of-Expert multilingual translation models, followed by finetuning on in-domain news data and noisy channel reranking. Compared to previous year's winning submissions, our multilingual system improved the translation quality on all language directions, with an average improvement of 2.0 BLEU. In the WMT2021 task, our system ranks first in 10 directions based on automatic evaluation.

Key facts. Cited by 57 papers in the map (data through 7 Oct 2026). Builds on 30 of its 40 references that are in the map. First posted on arXiv 6 Aug 2021.

Builds on

30 of 40 references.

References · 30

  1. 2017Attention Is All You Needcited by 59,932
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  3. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  4. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,973
  5. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,790
  6. 2020GShard: Scaling Giant Models with Conditional Computation and Automatic Shardingcited by 1,601
  7. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,486
  8. 2016Bag of Tricks for Efficient Text Classificationcited by 869
  9. 2015Improving Neural Machine Translation Models with Monolingual Datacited by 784
  10. 2020Language-agnostic BERT Sentence Embeddingcited by 621
  11. 2020Beyond English-Centric Multilingual Machine Translationcited by 559
  12. 2019CCNet: Extracting High Quality Monolingual Datasets from Web Crawl Datacited by 544
18 more
  1. 2020Multilingual Translation with Extensible Multilingual Pretraining and Finetuningcited by 287
  2. 2020Improving Massively Multilingual Neural Machine Translation and Zero-Shot Translationcited by 277
  3. 2019Massively Multilingual Neural Machine Translation in the Wild: Findings and Challengescited by 253
  4. 2019Massively Multilingual Neural Machine Translationcited by 246
  5. 2019Facebook FAIR's WMT19 News Translation Task Submissioncited by 179
  6. 2019CCMatrix: Mining Billions of High-Quality Parallel Sentences on the WEBcited by 177
  7. 2019CCAligned: A Massive Collection of Cross-Lingual Web-Document Pairscited by 136
  8. 2016Multi-Way, Multilingual Neural Machine Translation with a Shared Attention Mechanismcited by 117
  9. 2018Margin-based Parallel Corpus Mining with Multilingual Sentence Embeddingscited by 83
  10. 2020Cross-lingual Retrieval for Iterative Self-Supervised Trainingcited by 54
  11. 2019The Effect of Translationese in Machine Translation Test Setscited by 49
  12. 2019Multilingual Neural Machine Translation with Language Clusteringcited by 47
  13. 2019On The Evaluation of Machine Translation Systems Trained With Back-Translationcited by 40
  14. 2019Translationese in Machine Translation Evaluationcited by 38
  15. 2019Simple and Effective Noisy Channel Modeling for Neural Machine Translationcited by 37
  16. 2019Domain, Translationese and Noise in Synthetic Data for Neural Machine Translationcited by 26
  17. 2020Language Models not just for Pre-training: Fast Online Neural Noisy Channel Modelingcited by 6
  18. 2019Facebook AI's WAT19 Myanmar-English Translation Task Submissioncited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 7; 2022: 24; 2023: 15; 2024: 7; 2025: 3; 2026: 1.

57 papers.

Sort:

By area:

  1. 2022Prompting PaLM for Translation: Assessing Strategies and Performancecited by 144
  2. 2021High Quality Rather than High Model Probability: Minimum Bayes Risk Decoding with Neural Metricscited by 104
  3. 2022A Few Thousand Translations Go a Long Way! Leveraging Pre-trained Models for African News Translationcited by 97
  4. 2021Survey of Low-Resource Machine Translationcited by 92
  5. 2023The unreasonable effectiveness of few-shot learning for machine translationcited by 91
  6. 2022Looking for a Needle in a Haystack: A Comprehensive Study of Hallucinations in Neural Machine Translationcited by 60
  7. 2023Do GPTs Produce Less Literal Translations?cited by 30
  8. 2023Iterative Translation Refinement with Large Language Modelscited by 27
  9. 2022ODE Transformer: An Ordinary Differential Equation-Inspired Model for Sequence Generationcited by 26
  10. 2024Regularized Best-of-N Sampling with Minimum Bayes Risk Objective for Language Model Alignmentcited by 24
  11. 2023Augmenting Large Language Model Translators via Translation Memoriescited by 21
  12. 2022Cedille: A large autoregressive French language modelcited by 20
38 more
  1. 2021Tricks for Training Sparse Translation Modelscited by 17
  2. 2024Hyperparameter-Free Approach for Faster Minimum Bayes Risk Decodingcited by 15
  3. 2025Charting the Landscape of African NLP: Mapping Progress and Shaping the Road Aheadcited by 15
  4. 2021Alternative Input Signals Ease Transfer in Multilingual Machine Translationcited by 14
  5. 2022Efficient yet Competitive Speech Translation: FBK@IWSLT2022cited by 14
  6. 2022Consistent Human Evaluation of Machine Translation across Language Pairscited by 13
  7. 2022Revisiting Checkpoint Averaging for Neural Machine Translationcited by 13
  8. 2023Learning Language-Specific Layers for Multilingual Machine Translationcited by 13
  9. 2022Domain-Specific Text Generation for Machine Translationcited by 12
  10. 2022Discord Questions: A Computational Approach To Diversity Analysis in News Coveragecited by 11
  11. 2022A Call for Clarity in Beam Search: How It Works and When It Stopscited by 10
  12. 2023Target-Agnostic Gender-Aware Contrastive Learning for Mitigating Bias in Multilingual Machine Translationcited by 10
  13. 2022Vega-MT: The JD Explore Academy Translation System for WMT22cited by 9
  14. 2024Does Cross-Cultural Alignment Change the Commonsense Morality of Language Models?cited by 9
  15. 2023Neural Machine Translation of Clinical Text: An Empirical Investigation into Multilingual Pre-Trained Language Models and Transfer-Learningcited by 8
  16. 2022The first neural machine translation system for the Erzya languagecited by 7
  17. 2023Efficiency Pentathlon: A Standardized Arena for Efficiency Evaluationcited by 7
  18. 2024Bifurcated Attention: Accelerating Massively Parallel Decoding with Shared Prefixes in LLMscited by 7
  19. 2022Bridging Cross-Lingual Gaps During Leveraging the Multilingual Sequence-to-Sequence Pretraining for Text Generation and Understandingcited by 6
  20. 2022A Theory of Unsupervised Translation Motivated by Understanding Animal Communicationcited by 6
  21. 2026MITRA: A Large-Scale Parallel Corpus and Multilingual Pretrained Language Model for Machine Translation and Semantic Retrieval for P=ali, Sanskrit, Buddhist Chinese, and Tibetancited by 5
  22. 2021Amortized Noisy Channel Neural Machine Translationcited by 4
  23. 2021How Infinitely Wide Neural Networks Can Benefit from Multi-task Learning -- an Exact Macroscopic Characterizationcited by 4
  24. 2022On the Effectiveness of Automated Metrics for Text Generation Systemscited by 4
  25. 2023Accurate Knowledge Distillation with n-best Rerankingcited by 4
  26. 2022vec2text with Round-Trip Translationscited by 3
  27. 2022Examining Large Pre-Trained Language Models for Machine Translation: What You Don't Know About Itcited by 3
  28. 2022Investigating Massive Multilingual Pre-Trained Machine Translation Models for Clinical Domain via Transfer Learningcited by 3
  29. 2022Tackling Low-Resourced Sign Language Translation: UPC at WMT-SLT 22cited by 3
  30. 2023Designing and Evaluating Interfaces that Highlight News Coverage Diversity Using Discord Questionscited by 3
  31. 2024MQM-Chat: Multidimensional Quality Metrics for Chat Translationcited by 3
  32. 2021Analyzing the Limits of Self-Supervision in Handling Bias in Languagecited by 2
  33. 2022Refining Low-Resource Unsupervised Translation by Language Disentanglement of Multilingual Modelcited by 2
  34. 2023Automating Behavioral Testing in Machine Translationcited by 2
  35. 2023Dissecting In-Context Learning of Translations in GPTscited by 2
  36. 2022Syntax-based data augmentation for Hungarian-English machine translationcited by 1
  37. 2022Language Tokens: A Frustratingly Simple Approach Improves Zero-Shot Performance of Multilingual Translationcited by 1
  38. 2022Can Domains Be Transferred Across Languages in Multi-Domain Multilingual Neural Machine Translation?cited by 1

50 most cited of 57 All 57 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Multilingual Language Model Pretraining using Machine-translated Data
  2. 2024Multilingual Large Language Models: A Systematic Survey
  3. 2025Image-Text Relation Prediction for Multilingual Tweets
  4. 2022SpeechMatrix: A Large-Scale Mined Corpus of Multilingual Speech-to-Speech Translations
  5. 2024A Survey on Multilingual Large Language Models: Corpora, Alignment, and Bias
  6. 2025Reasoning-Guided Claim Normalization for Noisy Multilingual Social Media Posts
  7. 2025Redefining Machine Translation on Social Network Services with Large Language Models
  8. 2024Multilingual Large Language Model: A Survey of Resources, Taxonomy and Frontiers