mlsanityData to 7 Oct 2026

Boosting Unsupervised Machine Translation with Pseudo-Parallel Data

arXiv 2310.14262 ↗First posted 22 Oct 2023cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Even with the latest developments in deep learning and large-scale language modeling, the task of machine translation (MT) of low-resource languages remains a challenge. Neural MT systems can be trained in an unsupervised way without any translation resources but the quality lags behind, especially in truly low-resource conditions. We propose a training strategy that relies on pseudo-parallel sentence pairs mined from monolingual corpora in addition to synthetic sentence pairs back-translated from monolingual corpora. We experiment with different training schedules and reach an improvement of up to 14.5 BLEU points (English to Ukrainian) over a baseline trained on back-translated data only.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 31 of its 38 references that are in the map. First posted on arXiv 22 Oct 2023.

Builds on

31 of 38 references.

References · 31

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  4. 2020Language Models are Few-Shot Learnerscited by 32,199
  5. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  6. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  7. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
  8. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,790
  9. 2022No Language Left Behind: Scaling Human-Centered Machine Translationcited by 1,290
  10. 2019Cross-lingual Language Model Pretrainingcited by 1,088
  11. 2020Multilingual Denoising Pre-training for Neural Machine Translationcited by 1,028
  12. 2015Improving Neural Machine Translation Models with Monolingual Datacited by 784
19 more
  1. 2019How multilingual is Multilingual BERT?cited by 642
  2. 2018Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyondcited by 434
  3. 2019MASS: Masked Sequence to Sequence Pre-training for Language Generationcited by 345
  4. 2023How Good Are GPT Models at Machine Translation? A Comprehensive Evaluationcited by 344
  5. 2016FastText.zip: Compressing text classification modelscited by 331
  6. 2017Unsupervised Machine Translation Using Monolingual Corpora Onlycited by 272
  7. 2017Unsupervised Neural Machine Translationcited by 148
  8. 2018Phrase-Based & Neural Unsupervised Machine Translationcited by 138
  9. 2018Margin-based Parallel Corpus Mining with Multilingual Sentence Embeddingscited by 83
  10. 2020On the Language Neutrality of Pre-trained Multilingual Representationscited by 82
  11. 2020Cross-lingual Retrieval for Iterative Self-Supervised Trainingcited by 54
  12. 2018Unsupervised Statistical Machine Translationcited by 51
  13. 2019An Effective Approach to Unsupervised Machine Translationcited by 37
  14. 2018Filtering and Mining Parallel Data in a Joint Multilingual Spacecited by 32
  15. 2020A Multilingual View of Unsupervised Machine Translationcited by 23
  16. 2020Harnessing Multilinguality in Unsupervised Machine Translation for Rare Languagescited by 22
  17. 2021Multilingual Unsupervised Neural Machine Translation with Denoising Adapterscited by 22
  18. 2021Unsupervised Multilingual Sentence Embeddings for Parallel Corpus Miningcited by 9
  19. 2021Integrating Unsupervised Data Generation into Self-Supervised Neural Machine Translation for Low-Resource Languagescited by 4

Built on by

Not cited yet.