# A Character-Level Decoder without Explicit Segmentation for Neural Machine Translation

arXiv: https://arxiv.org/abs/1603.06147 · HTML: https://mlsanity.com/paper/1603.06147/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Mar 2016.

## Built on by (most cited first, 21 of 21)

- [ByT5: Towards a token-free future with pre-trained byte-to-byte models](https://mlsanity.com/paper/2105.13626/) (2021)
- [Language Model Tokenizers Introduce Unfairness Between Languages](https://mlsanity.com/paper/2305.15425/) (2023)
- [Between words and characters: A Brief History of Open-Vocabulary Modeling and Tokenization in NLP](https://mlsanity.com/paper/2112.10508/) (2021)
- [Why don't people use character-level machine translation?](https://mlsanity.com/paper/2110.08191/) (2021)
- [How Robust is Neural Machine Translation to Language Imbalance in Multilingual Tokenizer Training?](https://mlsanity.com/paper/2204.14268/) (2022)
- [An Empirical Study of Translation Hypothesis Ensembling with Large Language Models](https://mlsanity.com/paper/2310.11430/) (2023)
- [Are Character-level Translations Worth the Wait? Comparing ByT5 and mT5 for Machine Translation](https://mlsanity.com/paper/2302.14220/) (2023)
- [Tokenization and Morphology in Multilingual Language Models: A Comparative Analysis of mT5 and ByT5](https://mlsanity.com/paper/2410.11627/) (2024)
- [Neural OCR Post-Hoc Correction of Historical Corpora](https://mlsanity.com/paper/2102.00583/) (2021)
- [How to Split: the Effect of Word Segmentation on Gender Bias in Speech Translation](https://mlsanity.com/paper/2105.13782/) (2021)
- [On-the-Fly Fusion of Large Language Models and Machine Translation](https://mlsanity.com/paper/2311.08306/) (2023)
- [Frequency matters: Modeling irregular morphological patterns in Spanish with Transformers](https://mlsanity.com/paper/2410.21013/) (2024)
- [Neural Machine Translation for Code Generation](https://mlsanity.com/paper/2305.13504/) (2023)
- [FlowletFormer: Network Behavioral Semantic Aware Pre-training Model for Traffic Classification](https://mlsanity.com/paper/2508.19924/) (2025)
- [BanglaByT5: Byte-Level Modelling for Bangla](https://mlsanity.com/paper/2505.17102/) (2025)
- [Improving Language and Modality Transfer in Translation by Character-level Modeling](https://mlsanity.com/paper/2505.24561/) (2025)
- [The boundaries of meaning: a case study in neural machine translation](https://mlsanity.com/paper/2210.00613/) (2022)
- [CharSpan: Utilizing Lexical Similarity to Enable Zero-Shot Machine Translation for Extremely Low-resource Languages](https://mlsanity.com/paper/2305.05214/) (2023)
- [Sent2Matrix: Folding Character Sequences in Serpentine Manifolds for Two-Dimensional Sentence](https://mlsanity.com/paper/2103.08387/) (2021)
- [Character-level NMT and language similarity](https://mlsanity.com/paper/2308.04398/) (2023)
- [Analyzing Byte-Pair Encoding on Monophonic and Polyphonic Symbolic Music: A Focus on Musical Phrase Segmentation](https://mlsanity.com/paper/2410.01448/) (2024)

## Abstract (arXiv)

The existing machine translation systems, whether phrase-based or neural, have relied almost exclusively on word-level modelling with explicit segmentation. In this paper, we ask a fundamental question: can neural machine translation generate a character sequence without any explicit segmentation? To answer this question, we evaluate an attention-based encoder-decoder with a subword-level encoder and a character-level decoder on four language pairs--En-Cs, En-De, En-Ru and En-Fi-- using the parallel corpora from WMT'15. Our experiments show that the models with a character-level decoder outperform the ones with a subword-level decoder on all of the four language pairs. Furthermore, the ensembles of neural models with a character-level decoder outperform the state-of-the-art non-neural machine translation systems on En-Cs, En-De and En-Fi and perform comparably on En-Ru.