# Optimizing Segmentation Granularity for Neural Machine Translation

arXiv: https://arxiv.org/abs/1810.08641 · HTML: https://mlsanity.com/paper/1810.08641/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Oct 2018.

## Built on by (most cited first, 14 of 14)

- [Between words and characters: A Brief History of Open-Vocabulary Modeling and Tokenization in NLP](https://mlsanity.com/paper/2112.10508/) (2021)
- [Robust Open-Vocabulary Translation from Visual Text Representations](https://mlsanity.com/paper/2104.08211/) (2021)
- [Domain Adaptation and Multi-Domain Adaptation for Neural Machine Translation: A Survey](https://mlsanity.com/paper/2104.06951/) (2021)
- [Joint Optimization of Tokenization and Downstream Model](https://mlsanity.com/paper/2105.12410/) (2021)
- [Incorporating Context into Subword Vocabularies](https://mlsanity.com/paper/2210.07095/) (2022)
- [Subword Segmental Language Modelling for Nguni Languages](https://mlsanity.com/paper/2210.06525/) (2022)
- [Stop Taking Tokenizers for Granted: They Are Core Design Decisions in Large Language Models](https://mlsanity.com/paper/2601.13260/) (2026)
- [Second Language (Arabic) Acquisition of LLMs via Progressive Vocabulary Expansion](https://mlsanity.com/paper/2412.12310/) (2024)
- [Tackling Low-Resourced Sign Language Translation: UPC at WMT-SLT 22](https://mlsanity.com/paper/2212.01140/) (2022)
- [Transformer-Based Low-Resource Language Translation: A Study on Standard Bengali to Sylheti](https://mlsanity.com/paper/2510.18898/) (2025)
- [Downstream Task-Oriented Neural Tokenizer Optimization with Vocabulary Restriction as Post Processing](https://mlsanity.com/paper/2304.10808/) (2023)
- [Tokenization Preference for Human and Machine Learning Model: An Annotation Study](https://mlsanity.com/paper/2304.10813/) (2023)
- [Self-Vocabularizing Training for Neural Machine Translation](https://mlsanity.com/paper/2503.13837/) (2025)
- [An Empirical Study on Chinese Character Decomposition in Multiword Expression-Aware Neural Machine Translation](https://mlsanity.com/paper/2512.15556/) (2025)

## Abstract (arXiv)

In neural machine translation (NMT), it is has become standard to translate using subword units to allow for an open vocabulary and improve accuracy on infrequent words. Byte-pair encoding (BPE) and its variants are the predominant approach to generating these subwords, as they are unsupervised, resource-free, and empirically effective. However, the granularity of these subword units is a hyperparameter to be tuned for each language and task, using methods such as grid search. Tuning may be done inexhaustively or skipped entirely due to resource constraints, leading to sub-optimal performance. In this paper, we propose a method to automatically tune this parameter using only one training pass. We incrementally introduce new vocabulary online based on the held-out validation loss, beginning with smaller, general subwords and adding larger, more specific units over the course of training. Our method matches the results found with grid search, optimizing segmentation granularity without any additional training time. We also show benefits in training efficiency and performance improvements for rare words due to the way embeddings for larger units are incrementally constructed by combining those from smaller units.