# Fully Character-Level Neural Machine Translation without Explicit Segmentation

arXiv: https://arxiv.org/abs/1610.03017 · HTML: https://mlsanity.com/paper/1610.03017/

## Key facts

- Cited by 55 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Oct 2016.

## Built on by (most cited first, 30 of 55)

- [ByT5: Towards a token-free future with pre-trained byte-to-byte models](https://mlsanity.com/paper/2105.13626/) (2021)
- [CANINE: Pre-training an Efficient Tokenization-Free Encoder for Language Representation](https://mlsanity.com/paper/2103.06874/) (2021)
- [Language Model Tokenizers Introduce Unfairness Between Languages](https://mlsanity.com/paper/2305.15425/) (2023)
- [Charformer: Fast Character Transformers via Gradient-based Subword Tokenization](https://mlsanity.com/paper/2106.12672/) (2021)
- [Between words and characters: A Brief History of Open-Vocabulary Modeling and Tokenization in NLP](https://mlsanity.com/paper/2112.10508/) (2021)
- [Position Information in Transformers: An Overview](https://mlsanity.com/paper/2102.11090/) (2021)
- [Language Modelling with Pixels](https://mlsanity.com/paper/2207.06991/) (2022)
- [Multi-view Subword Regularization](https://mlsanity.com/paper/2103.08490/) (2021)
- [Counter-Interference Adapter for Multilingual Machine Translation](https://mlsanity.com/paper/2104.08154/) (2021)
- [CUTE: Measuring LLMs' Understanding of Their Tokens](https://mlsanity.com/paper/2409.15452/) (2024)
- [You should evaluate your language model on marginal likelihood over tokenisations](https://mlsanity.com/paper/2109.02550/) (2021)
- [Why don't people use character-level machine translation?](https://mlsanity.com/paper/2110.08191/) (2021)
- [Lego-MT: Learning Detachable Models for Massively Multilingual Machine Translation](https://mlsanity.com/paper/2212.10551/) (2022)
- [How Robust is Neural Machine Translation to Language Imbalance in Multilingual Tokenizer Training?](https://mlsanity.com/paper/2204.14268/) (2022)
- [Are Character-level Translations Worth the Wait? Comparing ByT5 and mT5 for Machine Translation](https://mlsanity.com/paper/2302.14220/) (2023)
- [Alternative Input Signals Ease Transfer in Multilingual Machine Translation](https://mlsanity.com/paper/2110.07804/) (2021)
- [Tokenization and Morphology in Multilingual Language Models: A Comparative Analysis of mT5 and ByT5](https://mlsanity.com/paper/2410.11627/) (2024)
- [Frequency-Aware Contrastive Learning for Neural Machine Translation](https://mlsanity.com/paper/2112.14484/) (2021)
- [MMTAfrica: Multilingual Machine Translation for African Languages](https://mlsanity.com/paper/2204.04306/) (2022)
- [Subword-Delimited Downsampling for Better Character-Level Translation](https://mlsanity.com/paper/2212.01304/) (2022)
- [Improving Arabic Diacritization by Learning to Diacritize and Translate](https://mlsanity.com/paper/2109.14150/) (2021)
- [How Deep Are the Fakes? Focusing on Audio Deepfake: A Survey](https://mlsanity.com/paper/2111.14203/) (2021)
- [Adapting to Non-Centered Languages for Zero-shot Multilingual Translation](https://mlsanity.com/paper/2209.04138/) (2022)
- [Improving Multilingual Neural Machine Translation System for Indic Languages](https://mlsanity.com/paper/2209.13279/) (2022)
- [HyperLoader: Integrating Hypernetwork-Based LoRA and Adapter Layers into Multi-Task Transformers for Sequence Labelling](https://mlsanity.com/paper/2407.01411/) (2024)
- [Noisy UGC Translation at the Character Level: Revisiting Open-Vocabulary Capabilities and Robustness of Char-Based Models](https://mlsanity.com/paper/2110.12552/) (2021)
- [Local Byte Fusion for Neural Machine Translation](https://mlsanity.com/paper/2205.11490/) (2022)
- [ParaTTS: Learning Linguistic and Prosodic Cross-sentence Information in Paragraph-based TTS](https://mlsanity.com/paper/2209.06484/) (2022)
- [Assessing the Importance of Frequency versus Compositionality for Subword-based Tokenization in NMT](https://mlsanity.com/paper/2306.01393/) (2023)
- [Backdoor Attack on Multilingual Machine Translation](https://mlsanity.com/paper/2404.02393/) (2024)

## Abstract (arXiv)

Most existing machine translation systems operate at the level of words, relying on explicit segmentation to extract tokens. We introduce a neural machine translation (NMT) model that maps a source character sequence to a target character sequence without any segmentation. We employ a character-level convolutional network with max-pooling at the encoder to reduce the length of source representation, allowing the model to be trained at a speed comparable to subword-level models while capturing local regularities. Our character-to-character model outperforms a recently proposed baseline with a subword-level encoder on WMT'15 DE-EN and CS-EN, and gives comparable performance on FI-EN and RU-EN. We then demonstrate that it is possible to share a single character-level encoder across multiple languages by training a model on a many-to-one translation task. In this multilingual setting, the character-level encoder significantly outperforms the subword-level encoder on all the language pairs. We observe that on CS-EN, FI-EN and RU-EN, the quality of the multilingual character-level translation even surpasses the models specifically trained on that language pair alone, both in terms of BLEU score and human judgment.