# MinGram: A Minimalist Unigram Tokenizer with High Compression and Competitive Morphological Alignment

arXiv: https://arxiv.org/abs/2606.27019 · HTML: https://mlsanity.com/paper/2606.27019/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 32 references that are in the map.
- First posted on arXiv 25 Jun 2026.

## Builds on (most cited first, 26 of 26)

- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing](https://mlsanity.com/paper/1808.06226/) (2018)
- [Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidates](https://mlsanity.com/paper/1804.10959/) (2018)
- [Byte Pair Encoding is Suboptimal for Language Model Pretraining](https://mlsanity.com/paper/2004.03720/) (2020)
- [Tokenization Is More Than Compression](https://mlsanity.com/paper/2402.18376/) (2024)
- [Nemotron-CLIMB: CLustering-based Iterative Data Mixture Bootstrapping for Language Model Pre-training](https://mlsanity.com/paper/2504.13161/) (2025)
- [SuperBPE: Space Travel for Language Models](https://mlsanity.com/paper/2503.13423/) (2025)
- [Why do language models perform worse for morphologically complex languages?](https://mlsanity.com/paper/2411.14198/) (2024)
- [Tokenization and the Noiseless Channel](https://mlsanity.com/paper/2306.16842/) (2023)
- [Fishing for Magikarp: Automatically Detecting Under-trained Tokens in Large Language Models](https://mlsanity.com/paper/2405.05417/) (2024)
- [Greed is All You Need: An Evaluation of Tokenizer Inference Methods](https://mlsanity.com/paper/2403.01289/) (2024)
- [Boundless Byte Pair Encoding: Breaking the Pre-tokenization Barrier](https://mlsanity.com/paper/2504.00178/) (2025)
- [BPE Gets Picky: Efficient Vocabulary Refinement During Tokenizer Training](https://mlsanity.com/paper/2409.04599/) (2024)
- [MorphBPE: A Morpho-Aware Tokenizer Bridging Linguistic Complexity for Efficient LLM Training Across Morphologies](https://mlsanity.com/paper/2502.00894/) (2025)
- [Evaluating Morphological Alignment of Tokenizers in 70 Languages](https://mlsanity.com/paper/2507.06378/) (2025)
- [Parity-Aware Byte-Pair Encoding: Improving Cross-lingual Fairness in Tokenization](https://mlsanity.com/paper/2508.04796/) (2025)
- [The University of Edinburgh's Neural MT Systems for WMT17](https://mlsanity.com/paper/1708.00726/) (2017)
- [Scaffold-BPE: Enhancing Byte Pair Encoding for Large Language Models with Simple and Effective Scaffold Token Removal](https://mlsanity.com/paper/2404.17808/) (2024)
- [Two Counterexamples to Tokenization and the Noiseless Channel](https://mlsanity.com/paper/2402.14614/) (2024)
- [BPE Stays on SCRIPT: Structured Encoding for Robust Multilingual Pretokenization](https://mlsanity.com/paper/2505.24689/) (2025)
- [Incorporating Context into Subword Vocabularies](https://mlsanity.com/paper/2210.07095/) (2022)
- [Beyond Text Compression: Evaluating Tokenizers Across Scales](https://mlsanity.com/paper/2506.03101/) (2025)
- [Rethinking Tokenization for Rich Morphology: The Dominance of Unigram over BPE and Morphological Alignment](https://mlsanity.com/paper/2508.08424/) (2025)
- [Which Pieces Does Unigram Tokenization Really Need?](https://mlsanity.com/paper/2512.12641/) (2025)
- [Tokenisation via Convex Relaxations](https://mlsanity.com/paper/2605.22821/) (2026)
- [Evaluating Morphological Plausibility of Subword Tokenization via Statistical Alignment with Morpho-Syntactic Features](https://mlsanity.com/paper/2601.18536/) (2026)

## Built on by (most cited first, 5 of 5)

- [TokEval: A Tokenizer Evaluation Suite](https://mlsanity.com/paper/2608.18062/) (2026)
- [Explicit Boundary Markers for Subword Vocabularies](https://mlsanity.com/paper/2608.08847/) (2026)
- [Dynamic-Programming-Guided Hierarchical BPE and Empirical Analysis of Vocabulary Pruning](https://mlsanity.com/paper/2609.06898/) (2026)
- [Counting and Min-Cost Encoding for Tokenization in Large Language Models](https://mlsanity.com/paper/2610.01127/) (2026)
- [Language-Specific Effects of Tokenizer Choice in Multilingual Language Models](https://mlsanity.com/paper/2610.12144/) (2026)

## Abstract (arXiv)

The Unigram tokenizer uses an elegant representation which makes it straightforward to edit vocabularies, but its training is comparatively heavy and complex. We introduce MinGram (Minimalist Unigram), which keeps the token-list representation but simplifies training using a BPE-derived seed vocabulary, Hard EM on a minimum-token path, and a single flat score-pruning step. This removes the suffix array, the forward-backward pass, and the iterative prune loop, leaving a procedure that requires little beyond tokenizer inference itself. By making token count the primary objective and using a Unigram score only as a tiebreak, MinGram keeps the compression of pure token-count methods while retaining much of the morphological alignment and downstream quality of probabilistic ones. Across six languages, MinGram compresses better than both BPE and standard Unigram, and a compression-oriented variant matches the strongest token-count compressors while retaining substantially higher morphological alignment. In controlled downstream language-model training, Unigram-family tokenizers, with MinGram among the best, consistently beat BPE in bits-per-byte.