# Achieving Open Vocabulary Neural Machine Translation with Hybrid Word-Character Models

arXiv: https://arxiv.org/abs/1604.00788 · HTML: https://mlsanity.com/paper/1604.00788/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Apr 2016.

## Built on by (most cited first, 18 of 18)

- [CANINE: Pre-training an Efficient Tokenization-Free Encoder for Language Representation](https://mlsanity.com/paper/2103.06874/) (2021)
- [Between words and characters: A Brief History of Open-Vocabulary Modeling and Tokenization in NLP](https://mlsanity.com/paper/2112.10508/) (2021)
- [Why don't people use character-level machine translation?](https://mlsanity.com/paper/2110.08191/) (2021)
- [The Foundations of Tokenization: Statistical and Computational Concerns](https://mlsanity.com/paper/2407.11606/) (2024)
- [Inducing Character-level Structure in Subword-based Language Models with Type-level Interchange Intervention Training](https://mlsanity.com/paper/2212.09897/) (2022)
- [Frequency-Aware Contrastive Learning for Neural Machine Translation](https://mlsanity.com/paper/2112.14484/) (2021)
- [Noisy UGC Translation at the Character Level: Revisiting Open-Vocabulary Capabilities and Robustness of Char-Based Models](https://mlsanity.com/paper/2110.12552/) (2021)
- [Low-resource Neural Machine Translation with Cross-modal Alignment](https://mlsanity.com/paper/2210.06716/) (2022)
- [Assessing the Importance of Frequency versus Compositionality for Subword-based Tokenization in NMT](https://mlsanity.com/paper/2306.01393/) (2023)
- [From SALAMANDRA to SALAMANDRATA: BSC Submission for WMT25 General Machine Translation Shared Task](https://mlsanity.com/paper/2508.12774/) (2025)
- [FlowletFormer: Network Behavioral Semantic Aware Pre-training Model for Traffic Classification](https://mlsanity.com/paper/2508.19924/) (2025)
- [Data Augmentation to Address Out-of-Vocabulary Problem in Low-Resource Sinhala-English Neural Machine Translation](https://mlsanity.com/paper/2205.08722/) (2022)
- [The boundaries of meaning: a case study in neural machine translation](https://mlsanity.com/paper/2210.00613/) (2022)
- [Leveraging knowledge graphs to update scientific word embeddings using latent semantic imputation](https://mlsanity.com/paper/2210.15358/) (2022)
- [SelfSeg: A Self-supervised Sub-word Segmentation Method for Neural Machine Translation](https://mlsanity.com/paper/2307.16400/) (2023)
- [Domain Adaptation for Arabic Machine Translation: The Case of Financial Texts](https://mlsanity.com/paper/2309.12863/) (2023)
- [A Statistician Teaches Deep Learning](https://mlsanity.com/paper/2102.01194/) (2021)
- [A New NMT Model for Translating Clinical Texts from English to Spanish](https://mlsanity.com/paper/2508.18607/) (2025)

## Abstract (arXiv)

Nearly all previous work on neural machine translation (NMT) has used quite restricted vocabularies, perhaps with a subsequent method to patch in unknown words. This paper presents a novel word-character solution to achieving open vocabulary NMT. We build hybrid systems that translate mostly at the word level and consult the character components for rare words. Our character-level recurrent neural networks compute source word representations and recover unknown target words when needed. The twofold advantage of such a hybrid approach is that it is much faster and easier to train than character-based ones; at the same time, it never produces unknown words as in the case of word-based models. On the WMT'15 English to Czech translation task, this hybrid approach offers an addition boost of +2.1-11.4 BLEU points over models that already handle unknown words. Our best system achieves a new state-of-the-art result with 20.7 BLEU score. We demonstrate that our character models can successfully learn to not only generate well-formed words for Czech, a highly-inflected language with a very complex vocabulary, but also build correct representations for English source words.