# Character-level Transformer-based Neural Machine Translation

arXiv: https://arxiv.org/abs/2005.11239 · HTML: https://mlsanity.com/paper/2005.11239/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 May 2020.

## Built on by (most cited first, 7 of 7)

- [Why don't people use character-level machine translation?](https://mlsanity.com/paper/2110.08191/) (2021)
- [Hybrid quantum-classical machine learning for generative chemistry and drug design](https://mlsanity.com/paper/2108.11644/) (2021)
- [PolyIPA -- Multilingual Phoneme-to-Grapheme Conversion Model](https://mlsanity.com/paper/2412.09102/) (2024)
- [Diverse Dance Synthesis via Keyframes with Transformer Controllers](https://mlsanity.com/paper/2207.05906/) (2022)
- [The boundaries of meaning: a case study in neural machine translation](https://mlsanity.com/paper/2210.00613/) (2022)
- [Search Intenion Network for Personalized Query Auto-Completion in E-Commerce](https://mlsanity.com/paper/2403.02609/) (2024)
- [Character-level NMT and language similarity](https://mlsanity.com/paper/2308.04398/) (2023)

## Abstract (arXiv)

Neural machine translation (NMT) is nowadays commonly applied at the subword level, using byte-pair encoding. A promising alternative approach focuses on character-level translation, which simplifies processing pipelines in NMT considerably. This approach, however, must consider relatively longer sequences, rendering the training process prohibitively expensive. In this paper, we discuss a novel, Transformer-based approach, that we compare, both in speed and in quality to the Transformer at subword and character levels, as well as previously developed character-level models. We evaluate our models on 4 language pairs from WMT'15: DE-EN, CS-EN, FI-EN and RU-EN. The proposed novel architecture can be trained on a single GPU and is 34% percent faster than the character-level Transformer; still, the obtained results are at least on par with it. In addition, our proposed model outperforms the subword-level model in FI-EN and shows close results in CS-EN. To stimulate further research in this area and close the gap with subword-level NMT, we make all our code and models publicly available.