mlsanityData to 7 Oct 2026

Multilingual Unsupervised Neural Machine Translation with Denoising Adapters

arXiv 2110.10472 ↗First posted 20 Oct 2021cs.CLcited by 22

Not on the map. Why?

Abstract (arXiv)

We consider the problem of multilingual unsupervised machine translation, translating to and from languages that only have monolingual data by using auxiliary parallel language pairs. For this problem the standard procedure so far to leverage the monolingual data is back-translation, which is computationally costly and hard to tune. In this paper we propose instead to use denoising adapters, adapter layers with a denoising objective, on top of pre-trained mBART-50. In addition to the modularity and flexibility of such an approach we show that the resulting translations are on-par with back-translating as measured by BLEU, and furthermore it allows adding unseen languages incrementally.

Key facts. Cited by 22 papers in the map (data through 7 Oct 2026). Builds on 32 of its 41 references that are in the map. First posted on arXiv 20 Oct 2021.

Builds on

32 of 41 references.

References · 32

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  4. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  5. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  6. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  7. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,973
  8. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,790
  9. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,486
  10. 2019Cross-lingual Language Model Pretrainingcited by 1,088
  11. 2020Multilingual Denoising Pre-training for Neural Machine Translationcited by 1,028
  12. 2020COMET: A Neural Framework for MT Evaluationcited by 1,024
20 more
  1. 2015Improving Neural Machine Translation Models with Monolingual Datacited by 784
  2. 2016Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translationcited by 467
  3. 2020MAD-X: An Adapter-Based Framework for Multi-Task Cross-Lingual Transfercited by 443
  4. 2020Multilingual Translation with Extensible Multilingual Pretraining and Finetuningcited by 287
  5. 2017Unsupervised Machine Translation Using Monolingual Corpora Onlycited by 272
  6. 2019Massively Multilingual Neural Machine Translation in the Wild: Findings and Challengescited by 253
  7. 2019Massively Multilingual Neural Machine Translationcited by 246
  8. 2019Simple, Scalable Adaptation for Neural Machine Translationcited by 239
  9. 2018Scaling Neural Machine Translationcited by 204
  10. 2017Unsupervised Neural Machine Translationcited by 148
  11. 2018When and Why are Pre-trained Word Embeddings Useful for Neural Machine Translation?cited by 125
  12. 2016Multi-Way, Multilingual Neural Machine Translation with a Shared Attention Mechanismcited by 117
  13. 2020UDapter: Language Adaptation for Truly Universal Dependency Parsingcited by 77
  14. 2016Toward Multilingual Neural Machine Translation with Universal Encoder and Decodercited by 76
  15. 2017The IIT Bombay English-Hindi Parallel Corpuscited by 74
  16. 2019An Effective Approach to Unsupervised Machine Translationcited by 37
  17. 2020Recipes for Adapting Pre-trained Monolingual and Multilingual Models to Machine Translationcited by 28
  18. 2020Harnessing Multilinguality in Unsupervised Machine Translation for Rare Languagescited by 22
  19. 2020Knowledge Distillation for Multilingual Unsupervised Neural Machine Translationcited by 19
  20. 2020Reference Language based Unsupervised Neural Machine Translationcited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 11; 2023: 6; 2024: 2; 2025: 1; 2026: 1.

22 papers.

Sort:

By area:

  1. 2022Lifting the Curse of Multilinguality by Pre-training Modular Transformerscited by 145
  2. 2022BLOOM+1: Adding Language Support to BLOOM for Zero-Shot Promptingcited by 90
  3. 2023Modular Deep Learningcited by 82
  4. 2022u-HuBERT: Unified Mixed-Modal Speech Pretraining And Zero-Shot Transfer to Unlabeled Modalitycited by 36
  5. 2021Efficient Hierarchical Domain Adaptation for Pretrained Language Modelscited by 32
  6. 2023mmT5: Modular Multilingual Pre-Training Solves Source Language Hallucinationscited by 24
  7. 2022Hyper-X: A Unified Hypernetwork for Multi-Task Multilingual Transfercited by 22
  8. 2022Language-Family Adapters for Low-Resource Multilingual Neural Machine Translationcited by 14
  9. 2022Multilingual Pre-training with Language and Task Adaptation for Multilingual Text Style Transfercited by 8
  10. 2023Communication Efficient Federated Learning for Multilingual Neural Machine Translation with Adaptercited by 8
  11. 2022When does Parameter-Efficient Transfer Learning Work for Machine Translation?cited by 7
  12. 2022Subword-Delimited Downsampling for Better Character-Level Translationcited by 7
10 more
  1. 2022Adapting BigScience Multilingual Model to Unseen Languagescited by 6
  2. 2022Flow-Adapter Architecture for Unsupervised Machine Translationcited by 4
  3. 2024Unlocking Parameter-Efficient Fine-Tuning for Low-Resource Language Translationcited by 4
  4. 2023Unlikelihood Tuning on Negative Samples Amazingly Improves Zero-Shot Translationcited by 2
  5. 2023Extrapolating Multilingual Understanding Models as Multilingual Generatorscited by 1
  6. 2026HEBATRON: A Hebrew-Specialized Open-Weight Mixture-of-Experts Language Modelcited by 1
  7. 2022Use of Transformer-Based Models for Word-Level Transliteration of the Book of the Dean of Lismorecited by 0
  8. 2023Boosting Unsupervised Machine Translation with Pseudo-Parallel Datacited by 0
  9. 2024Cross-lingual Back-Parsing: Utterance Synthesis from Meaning Representation for Zero-Resource Semantic Parsingcited by 0
  10. 2025Limited-Resource Adapters Are Regularizers, Not Linguistscited by 0