The Impact of Syntactic and Semantic Proximity on Machine Translation with Back-Translation
Not on the map. Why?
Abstract (arXiv)
Unsupervised on-the-fly back-translation, in conjunction with multilingual pretraining, is the dominant method for unsupervised neural machine translation. Theoretically, however, the method should not work in general. We therefore conduct controlled experiments with artificial languages to determine what properties of languages make back-translation an effective training method, covering lexical, syntactic, and semantic properties. We find, contrary to popular belief, that (i) parallel word frequency distributions, (ii) partially shared vocabulary, and (iii) similar syntactic structure across languages are not sufficient to explain the success of back-translation. We show however that even crude semantic signal (similar lexical fields across languages) does improve alignment of two languages through back-translation. We conjecture that rich semantic dependencies, parallel across languages, are at the root of the success of unsupervised methods based on back-translation. Overall, the success of unsupervised machine translation was far from being analytically guaranteed. Instead, it is another proof that languages of the world share deep similarities, and we hope to show how to identify which of these similarities can serve the development of unsupervised, cross-linguistic tools.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 40 of its 60 references that are in the map. First posted on arXiv 26 Mar 2024.
Builds on
40 of 60 references.
References · 40
- 2017Attention Is All You Needcited by 60,361
- 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
- 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
- 2014Sequence to Sequence Learning with Neural Networkscited by 2,986
- 2016Enriching Word Vectors with Subword Informationcited by 1,583
- 2016Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translationcited by 1,475
- 2020BLEURT: Learning Robust Metrics for Text Generationcited by 1,203
- 2019Cross-lingual Language Model Pretrainingcited by 1,090
- 2020Multilingual Denoising Pre-training for Neural Machine Translationcited by 1,035
- 2020COMET: A Neural Framework for MT Evaluationcited by 1,033
- 2015Improving Neural Machine Translation Models with Monolingual Datacited by 788
- 2017Word Translation Without Parallel Datacited by 487
28 more
- 2019MASS: Masked Sequence to Sequence Pre-training for Language Generationcited by 347
- 2019Beto, Bentz, Becas: The Surprising Cross-Lingual Effectiveness of BERTcited by 301
- 2017Unsupervised Machine Translation Using Monolingual Corpora Onlycited by 273
- 2013Exploiting Similarities among Languages for Machine Translationcited by 191
- 2019Cross-Lingual Ability of Multilingual BERT: An Empirical Studycited by 182
- 2019Emerging Cross-lingual Structure in Pretrained Language Modelscited by 181
- 2017Unsupervised Neural Machine Translationcited by 149
- 2016Dual Learning for Machine Translationcited by 148
- 2018Phrase-Based & Neural Unsupervised Machine Translationcited by 138
- 2021Survey of Low-Resource Machine Translationcited by 92
- 2019Tagged Back-Translationcited by 77
- 2019On The Evaluation of Machine Translation Systems Trained With Back-Translationcited by 40
- 2021Examining the Inductive Bias of Neural Language Models with Artificial Languagescited by 40
- 2016Semi-Supervised Learning for Neural Machine Translationcited by 36
- 2019Pre-trained Language Model Representations for Language Generationcited by 34
- 2020When and Why is Unsupervised Neural Machine Translation Useless?cited by 34
- 2017Dual Supervised Learningcited by 33
- 2022UniTE: Unified Translation Evaluationcited by 28
- 2019Effective Cross-lingual Transfer of Neural Machine Translation Models without Shared Vocabulariescited by 27
- 2020Dynamic Data Selection and Weighting for Iterative Back-Translationcited by 25
- 2021On the Transferability of Pre-trained Language Models: A Study from Artificial Datasetscited by 25
- 2021BLEU, METEOR, BERTScore: Evaluation of Metrics Performance in Assessing Critical Translation Errors in Sentiment-oriented Textcited by 22
- 2018Unsupervised Neural Machine Translation with Weight Sharingcited by 15
- 2022On the Complementarity between Pre-Training and Random-Initialization for Resource-Rich Machine Translationcited by 15
- 2018Investigating Backtranslation in Neural Machine Translationcited by 14
- 2018Bi-Directional Neural Machine Translation with Synthetic Parallel Datacited by 11
- 2019Translating Translationese: A Two-Step Approach to Unsupervised Machine Translationcited by 11
- 2018Explaining and Generalizing Back-Translation through Wake-Sleepcited by 7
Built on by
Citing papers per year: 2025: 2.
2 papers.