# SuTRA : Structurally-Unified Tokenization with Root Awareness

arXiv: https://arxiv.org/abs/2608.18087 · HTML: https://mlsanity.com/paper/2608.18087/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 29 of its 57 references that are in the map.
- First posted on arXiv 5 Jun 2026.

## Builds on (most cited first, 29 of 29)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [Gemma 3 Technical Report](https://mlsanity.com/paper/2503.19786/) (2025)
- [fairseq: A Fast, Extensible Toolkit for Sequence Modeling](https://mlsanity.com/paper/1904.01038/) (2019)
- [Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidates](https://mlsanity.com/paper/1804.10959/) (2018)
- [ByT5: Towards a token-free future with pre-trained byte-to-byte models](https://mlsanity.com/paper/2105.13626/) (2021)
- [CANINE: Pre-training an Efficient Tokenization-Free Encoder for Language Representation](https://mlsanity.com/paper/2103.06874/) (2021)
- [BPE-Dropout: Simple and Effective Subword Regularization](https://mlsanity.com/paper/1910.13267/) (2019)
- [Charformer: Fast Character Transformers via Gradient-based Subword Tokenization](https://mlsanity.com/paper/2106.12672/) (2021)
- [Towards Leaving No Indic Language Behind: Building Monolingual Corpora, Benchmark and Models for Indic Languages](https://mlsanity.com/paper/2212.05409/) (2022)
- [SuperBPE: Space Travel for Language Models](https://mlsanity.com/paper/2503.13423/) (2025)
- [Impact of Tokenization on Language Models: An Analysis for Turkish](https://mlsanity.com/paper/2204.08832/) (2022)
- [PaperBanana: Automating Academic Illustration for AI Scientists](https://mlsanity.com/paper/2601.23265/) (2026)
- [MorphBPE: A Morpho-Aware Tokenizer Bridging Linguistic Complexity for Efficient LLM Training Across Morphologies](https://mlsanity.com/paper/2502.00894/) (2025)
- [Evaluating Morphological Alignment of Tokenizers in 70 Languages](https://mlsanity.com/paper/2507.06378/) (2025)
- [TransLIST: A Transformer-Based Linguistically Informed Sanskrit Tokenizer](https://mlsanity.com/paper/2210.11753/) (2022)
- [Tokenization Matters: Improving Zero-Shot NER for Indic Languages](https://mlsanity.com/paper/2504.16977/) (2025)
- [UniMorph 4.0: Universal Morphology](https://mlsanity.com/paper/2205.03608/) (2022)
- [Evaluating Tokenizer Performance of Large Language Models Across Official Indian Languages](https://mlsanity.com/paper/2411.12240/) (2024)
- [T-FREE: Subword Tokenizer-Free Generative LLMs via Sparse Representations for Memory-Efficient Embeddings](https://mlsanity.com/paper/2406.19223/) (2024)
- [MorphPiece : A Linguistic Tokenizer for Large Language Models](https://mlsanity.com/paper/2307.07262/) (2023)
- [MorphTok: Morphologically Grounded Tokenization for Indian Languages](https://mlsanity.com/paper/2504.10335/) (2025)
- [LBPE: Long-token-first Tokenization to Improve Large Language Models](https://mlsanity.com/paper/2411.05504/) (2024)
- [BabyLM Challenge: Exploring the Effect of Variation Sets on Language Model Training Efficiency](https://mlsanity.com/paper/2411.09587/) (2024)
- [SampoNLP: A Self-Referential Toolkit for Morphological Analysis of Subword Tokenizers](https://mlsanity.com/paper/2601.04469/) (2026)
- [Is Sanskrit the most token-efficient language? A quantitative study using GPT, Gemini, and SentencePiece](https://mlsanity.com/paper/2601.06142/) (2026)

## Abstract (arXiv)

Existing subword tokenizers optimize statistical compression but ignore morphological structure, particularly the relationship between roots and affixes. This is harmful for morphologically rich Indic languages, where basic units are complex orthographic syllables (aksharas) rather than letters. Frequency-based methods over-fragment words, arbitrarily splitting roots and affixes - a phenomenon we term Morphological Shattering. We propose SuTRA (Structurally-Unified Tokenization with Root Awareness), a morphology-aware algorithm that preserves akshara indivisibility and penalizes merges crossing morphological boundaries. We also release a new morphological segmentation dataset for Hindi, Marathi, and Gujarati. SuTRA reduces shattering, achieving peak gains of +14.7% in morphological alignment (Boundary F1) and +34% in semantic recoverability (Hindi) over BPE. These structural gains yield an average improvement of +8.08 chrF2 in machine translation.