# The Impact of Preprocessing on Arabic-English Statistical and Neural Machine Translation

arXiv: https://arxiv.org/abs/1906.11751 · HTML: https://mlsanity.com/paper/1906.11751/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Jun 2019.

## Built on by (most cited first, 6 of 6)

- [Evaluating Various Tokenizers for Arabic Text Classification](https://mlsanity.com/paper/2106.07540/) (2021)
- [Investigating Code-Mixed Modern Standard Arabic-Egyptian to English Machine Translation](https://mlsanity.com/paper/2105.13573/) (2021)
- [Investigating Lexical Replacements for Arabic-English Code-Switched Data Augmentation](https://mlsanity.com/paper/2205.12649/) (2022)
- [Exploring Segmentation Approaches for Neural Machine Translation of Code-Switched Egyptian Arabic-English Text](https://mlsanity.com/paper/2210.06990/) (2022)
- [Exploring Tokenization Strategies and Vocabulary Sizes for Enhanced Arabic Language Models](https://mlsanity.com/paper/2403.11130/) (2024)
- [Domain Adaptation for Arabic Machine Translation: The Case of Financial Texts](https://mlsanity.com/paper/2309.12863/) (2023)

## Abstract (arXiv)

Neural networks have become the state-of-the-art approach for machine translation (MT) in many languages. While linguistically-motivated tokenization techniques were shown to have significant effects on the performance of statistical MT, it remains unclear if those techniques are well suited for neural MT. In this paper, we systematically compare neural and statistical MT models for Arabic-English translation on data preprecossed by various prominent tokenization schemes. Furthermore, we consider a range of data and vocabulary sizes and compare their effect on both approaches. Our empirical results show that the best choice of tokenization scheme is largely based on the type of model and the size of data. We also show that we can gain significant improvements using a system selection that combines the output from neural and statistical MT.