# A Comparison of Transformer and Recurrent Neural Networks on Multilingual Neural Machine Translation

arXiv: https://arxiv.org/abs/1806.06957 · HTML: https://mlsanity.com/paper/1806.06957/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Jun 2018.

## Built on by (most cited first, 23 of 23)

- [Neural Machine Translation for Low-Resource Languages: A Survey](https://mlsanity.com/paper/2106.15115/) (2021)
- [Multilingual training for Software Engineering](https://mlsanity.com/paper/2112.02043/) (2021)
- [A Survey on Low-Resource Neural Machine Translation](https://mlsanity.com/paper/2107.04239/) (2021)
- [MMTAfrica: Multilingual Machine Translation for African Languages](https://mlsanity.com/paper/2204.04306/) (2022)
- [Learning from flowsheets: A generative transformer model for autocompletion of flowsheets](https://mlsanity.com/paper/2208.00859/) (2022)
- [Leveraging Long-Context Large Language Models for Multi-Document Understanding and Summarization in Enterprise Applications](https://mlsanity.com/paper/2409.18454/) (2024)
- [A transformer-based model for default prediction in mid-cap corporate markets](https://mlsanity.com/paper/2111.09902/) (2021)
- [Speech-to-SQL: Towards Speech-driven SQL Query Generation From Natural Language Question](https://mlsanity.com/paper/2201.01209/) (2022)
- [An analysis of vaccine-related sentiments from development to deployment of COVID-19 vaccines](https://mlsanity.com/paper/2306.13797/) (2023)
- [Towards Tailored Recovery of Lexical Diversity in Literary Machine Translation](https://mlsanity.com/paper/2408.17308/) (2024)
- [Transformers versus LSTMs for electronic trading](https://mlsanity.com/paper/2309.11400/) (2023)
- [Improving Zero-shot Neural Machine Translation on Language-specific Encoders-Decoders](https://mlsanity.com/paper/2102.06578/) (2021)
- [CUNI systems for WMT21: Multilingual Low-Resource Translation for Indo-European Languages Shared Task](https://mlsanity.com/paper/2109.09354/) (2021)
- [Transfer Learning for Multi-lingual Tasks -- a Survey](https://mlsanity.com/paper/2110.02052/) (2021)
- [EAG: Extract and Generate Multi-way Aligned Corpus for Complete Multi-lingual Neural Machine Translation](https://mlsanity.com/paper/2203.02180/) (2022)
- [A Case Report On The "A.I. Locked-In Problem": social concerns with modern NLP](https://mlsanity.com/paper/2209.12687/) (2022)
- [Learning to Generate 3D Representations of Building Roofs Using Single-View Aerial Imagery](https://mlsanity.com/paper/2303.11215/) (2023)
- [Curated Datasets and Neural Models for Machine Translation of Informal Registers between Mayan and Spanish Vernaculars](https://mlsanity.com/paper/2404.07673/) (2024)
- [An empirical analysis of phrase-based and neural machine translation](https://mlsanity.com/paper/2103.03108/) (2021)
- [An exploratory experiment on Hindi, Bengali hate-speech detection and transfer learning using neural networks](https://mlsanity.com/paper/2201.01997/) (2022)
- [Are Deep Sequence Classifiers Good at Non-Trivial Generalization?](https://mlsanity.com/paper/2210.13082/) (2022)
- [Automatic detection of problem-gambling signs from online texts using large language models](https://mlsanity.com/paper/2312.00804/) (2023)
- [Biomedical Machine Translation for Low-Resource Arabic-Script Languages via Cross-Lingual Transfer and LoRA Adapter Merging](https://mlsanity.com/paper/2607.22300/) (2026)

## Abstract (arXiv)

Recently, neural machine translation (NMT) has been extended to multilinguality, that is to handle more than one translation direction with a single system. Multilingual NMT showed competitive performance against pure bilingual systems. Notably, in low-resource settings, it proved to work effectively and efficiently, thanks to shared representation space that is forced across languages and induces a sort of transfer-learning. Furthermore, multilingual NMT enables so-called zero-shot inference across language pairs never seen at training time. Despite the increasing interest in this framework, an in-depth analysis of what a multilingual NMT model is capable of and what it is not is still missing. Motivated by this, our work (i) provides a quantitative and comparative analysis of the translations produced by bilingual, multilingual and zero-shot systems; (ii) investigates the translation quality of two of the currently dominant neural architectures in MT, which are the Recurrent and the Transformer ones; and (iii) quantitatively explores how the closeness between languages influences the zero-shot translation. Our analysis leverages multiple professional post-edits of automatic translations by several different systems and focuses both on automatic standard metrics (BLEU and TER) and on widely used error categories, which are lexical, morphology, and word order errors.