# The Token Tax: Systematic Bias in Multilingual Tokenization

arXiv: https://arxiv.org/abs/2509.05486 · HTML: https://mlsanity.com/paper/2509.05486/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 19 references that are in the map.
- First posted on arXiv 5 Sep 2025.

## Builds on (most cited first, 13 of 13)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [The State and Fate of Linguistic Diversity and Inclusion in the NLP World](https://mlsanity.com/paper/2004.09095/) (2020)
- [How Good is Your Tokenizer? On the Monolingual Performance of Multilingual Language Models](https://mlsanity.com/paper/2012.15613/) (2020)
- [Language Model Tokenizers Introduce Unfairness Between Languages](https://mlsanity.com/paper/2305.15425/) (2023)
- [Do All Languages Cost the Same? Tokenization in the Era of Commercial Language Models](https://mlsanity.com/paper/2305.13707/) (2023)
- [On The Computational Complexity of Self-Attention](https://mlsanity.com/paper/2209.04881/) (2022)
- [Tokenizer Choice For LLM Training: Negligible or Crucial?](https://mlsanity.com/paper/2310.08754/) (2023)
- [IrokoBench: A New Benchmark for African Languages in the Age of Large Language Models](https://mlsanity.com/paper/2406.03368/) (2024)
- [MAGNET: Improving the Multilingual Fairness of Language Models with Adaptive Gradient-Based Tokenization](https://mlsanity.com/paper/2407.08818/) (2024)
- [Where Are We? Evaluating LLM Performance on African Languages](https://mlsanity.com/paper/2502.19582/) (2025)
- [FLEXITOKENS: Flexible Tokenization for Evolving Language Models](https://mlsanity.com/paper/2507.12720/) (2025)
- [mSTEB: Massively Multilingual Evaluation of LLMs on Speech and Text Tasks](https://mlsanity.com/paper/2506.08400/) (2025)
- [Local Byte Fusion for Neural Machine Translation](https://mlsanity.com/paper/2205.11490/) (2022)

## Built on by (most cited first, 14 of 14)

- [The Roots of Performance Disparity in Multilingual Language Models: Intrinsic Modeling Difficulty or Design Choices?](https://mlsanity.com/paper/2601.07220/) (2026)
- [TukaBench: A Culturally Grounded Jailbreak Benchmark for African Languages](https://mlsanity.com/paper/2606.01322/) (2026)
- [Tokenization with Split Trees](https://mlsanity.com/paper/2605.22705/) (2026)
- [Phonemes to the Rescue: Multilingual Tokenization Based on International Phonetic Alphabet](https://mlsanity.com/paper/2606.20993/) (2026)
- [TokEval: A Tokenizer Evaluation Suite](https://mlsanity.com/paper/2608.18062/) (2026)
- [Multilinguality at the Edge: Developing Language Models for the Global South](https://mlsanity.com/paper/2604.21637/) (2026)
- [AmchiBias: Measuring Stereotypical Bias in Goan Identity Groups with a Minimal Pair Dataset in English and Konkani](https://mlsanity.com/paper/2606.15191/) (2026)
- [Why do Large Language Models Fail in Low-resource Translation? Unraveling the Token Dynamics of Large Language Models for Machine Translation](https://mlsanity.com/paper/2605.07533/) (2026)
- [Modular Monolingual Adaptation using Pretrained Language Models](https://mlsanity.com/paper/2606.06738/) (2026)
- [Surpassing Scale by Efficiency: A Compact 135M Parameter Foundational LLM Natively Adapted for the Bangla Language](https://mlsanity.com/paper/2606.16383/) (2026)
- [The Role of Prompt Language and Translation-Theory-Driven Prompts in Large Language Models: A Case Study on Spanish-Chinese Journalistic Translation](https://mlsanity.com/paper/2607.03160/) (2026)
- [Removable and Irreducible: A Token-Cost Ledger for the Multilingual Tokenization Tax](https://mlsanity.com/paper/2609.00378/) (2026)
- [What Does Post-Training Change in Multilingual Reasoning?](https://mlsanity.com/paper/2609.37104/) (2026)
- [The Invisible Language Tax: Token Premiums of French and Regional Languages in 2026 LLM Tokenizers, and a French-Optimized Prototype](https://mlsanity.com/paper/2609.39001/) (2026)

## Abstract (arXiv)

Tokenization inefficiency imposes structural disadvantages on morphologically complex, low-resource languages, inflating compute resources and depressing accuracy. We evaluate 10 large language models (LLMs) on AfriMMLU (9,000 MCQA items; 5 subjects; 16 African languages) and show that fertility (tokens/word) reliably predicts accuracy. Higher fertility consistently predicts lower accuracy across all models and subjects. We further find that reasoning models (DeepSeek, o1) consistently outperform non-reasoning peers across high and low resource languages in the AfriMMLU dataset, narrowing accuracy gaps observed in prior generations. Finally, translating token inflation to economics, a doubling in tokens results in quadrupled training cost and time, underscoring the token tax faced by many languages. These results motivate morphologically aware tokenization, fair pricing, and multilingual benchmarks for equitable natural language processing (NLP).