# Character-based Neural Machine Translation

arXiv: https://arxiv.org/abs/1603.00810 · HTML: https://mlsanity.com/paper/1603.00810/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Mar 2016.

## Built on by (most cited first, 30 of 33)

- [ByT5: Towards a token-free future with pre-trained byte-to-byte models](https://mlsanity.com/paper/2105.13626/) (2021)
- [Tokenization and the Noiseless Channel](https://mlsanity.com/paper/2306.16842/) (2023)
- [A Survey on Green Deep Learning](https://mlsanity.com/paper/2111.05193/) (2021)
- [Domain Adaptation and Multi-Domain Adaptation for Neural Machine Translation: A Survey](https://mlsanity.com/paper/2104.06951/) (2021)
- [The Effectiveness of Morphology-aware Segmentation in Low-Resource Neural Machine Translation](https://mlsanity.com/paper/2103.11189/) (2021)
- [Scaffold-BPE: Enhancing Byte Pair Encoding for Large Language Models with Simple and Effective Scaffold Token Removal](https://mlsanity.com/paper/2404.17808/) (2024)
- [Are Character-level Translations Worth the Wait? Comparing ByT5 and mT5 for Machine Translation](https://mlsanity.com/paper/2302.14220/) (2023)
- [Alternative Input Signals Ease Transfer in Multilingual Machine Translation](https://mlsanity.com/paper/2110.07804/) (2021)
- [Towards energy-efficient Deep Learning: An overview of energy-efficient approaches along the Deep Learning Lifecycle](https://mlsanity.com/paper/2303.01980/) (2023)
- [Neural OCR Post-Hoc Correction of Historical Corpora](https://mlsanity.com/paper/2102.00583/) (2021)
- [A Vocabulary-Free Multilingual Neural Tokenizer for End-to-End Task Learning](https://mlsanity.com/paper/2204.10815/) (2022)
- [How to Split: the Effect of Word Segmentation on Gender Bias in Speech Translation](https://mlsanity.com/paper/2105.13782/) (2021)
- [Lookup-Table Recurrent Language Models for Long Tail Speech Recognition](https://mlsanity.com/paper/2104.04552/) (2021)
- [Subword-Delimited Downsampling for Better Character-Level Translation](https://mlsanity.com/paper/2212.01304/) (2022)
- [Neural Machine Translation for the Indigenous Languages of the Americas: An Introduction](https://mlsanity.com/paper/2306.06804/) (2023)
- [Embedding API Dependency Graph for Neural Code Generation](https://mlsanity.com/paper/2103.15361/) (2021)
- [Noisy UGC Translation at the Character Level: Revisiting Open-Vocabulary Capabilities and Robustness of Char-Based Models](https://mlsanity.com/paper/2110.12552/) (2021)
- [Frequency matters: Modeling irregular morphological patterns in Spanish with Transformers](https://mlsanity.com/paper/2410.21013/) (2024)
- [SWE2: SubWord Enriched and Significant Word Emphasized Framework for Hate Speech Detection](https://mlsanity.com/paper/2409.16673/) (2024)
- [How Effective is Byte Pair Encoding for Out-Of-Vocabulary Words in Neural Machine Translation?](https://mlsanity.com/paper/2208.05225/) (2022)
- [Neural Machine Translation for Code Generation](https://mlsanity.com/paper/2305.13504/) (2023)
- [Recurrent Neural Networks with Mixed Hierarchical Structures for Natural Language Processing](https://mlsanity.com/paper/2106.02562/) (2021)
- [Piecewise-constant Neural ODEs](https://mlsanity.com/paper/2106.06621/) (2021)
- [BanglaByT5: Byte-Level Modelling for Bangla](https://mlsanity.com/paper/2505.17102/) (2025)
- [Improving Language and Modality Transfer in Translation by Character-level Modeling](https://mlsanity.com/paper/2505.24561/) (2025)
- [The boundaries of meaning: a case study in neural machine translation](https://mlsanity.com/paper/2210.00613/) (2022)
- [SelfSeg: A Self-supervised Sub-word Segmentation Method for Neural Machine Translation](https://mlsanity.com/paper/2307.16400/) (2023)
- [Domain Adaptation for Arabic Machine Translation: The Case of Financial Texts](https://mlsanity.com/paper/2309.12863/) (2023)
- [Larth: Dataset and Machine Translation for Etruscan](https://mlsanity.com/paper/2310.05688/) (2023)
- [Self-Vocabularizing Training for Neural Machine Translation](https://mlsanity.com/paper/2503.13837/) (2025)

## Abstract (arXiv)

Neural Machine Translation (MT) has reached state-of-the-art results. However, one of the main challenges that neural MT still faces is dealing with very large vocabularies and morphologically rich languages. In this paper, we propose a neural MT system using character-based embeddings in combination with convolutional and highway layers to replace the standard lookup-based word representations. The resulting unlimited-vocabulary and affix-aware source word embeddings are tested in a state-of-the-art neural MT based on an attention-based bidirectional recurrent neural network. The proposed MT scheme provides improved results even when the source language is not morphologically rich. Improvements up to 3 BLEU points are obtained in the German-English WMT task.