# Morpheus: A Morphology-Aware Neural Tokenizer and Word Embedder for Turkish

arXiv: https://arxiv.org/abs/2606.18717 · HTML: https://mlsanity.com/paper/2606.18717/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 19 references that are in the map.
- First posted on arXiv 17 Jun 2026.

## Builds on (most cited first, 12 of 12)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoFormer: Enhanced Transformer with Rotary Position Embedding](https://mlsanity.com/paper/2104.09864/) (2021)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing](https://mlsanity.com/paper/1808.06226/) (2018)
- [Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidates](https://mlsanity.com/paper/1804.10959/) (2018)
- [Impact of Tokenization on Language Models: An Analysis for Turkish](https://mlsanity.com/paper/2204.08832/) (2022)
- [Tokenization Standards for Linguistic Integrity: Turkish as a Benchmark](https://mlsanity.com/paper/2502.07057/) (2025)
- [Tokens with Meaning: A Hybrid Tokenization Approach for Turkish](https://mlsanity.com/paper/2508.14292/) (2025)
- [Dougal Dil .Icslemede Tokenizasyon Standartlari ve "Olcc"um"u: T"urkcce "Uzerinden B"uy"uk Dil Modellerinin Karcsilacstirmali Analizi](https://mlsanity.com/paper/2508.13058/) (2025)
- [Optimal Turkish Subword Strategies at Scale: Systematic Evaluation of Data, Vocabulary, Morphology Interplay](https://mlsanity.com/paper/2602.06942/) (2026)
- [HeceTokenizer: A Syllable-Based Tokenization Approach for Turkish Retrieval](https://mlsanity.com/paper/2604.10665/) (2026)
- [Adapting Multilingual Embedding Models to Turkish via Cross-Lingual Tokenizer Surgery and Offline Distillation](https://mlsanity.com/paper/2605.29992/) (2026)

## Abstract (arXiv)

Turkish is agglutinative: meaning is carried by morphemes, yet the subword tokenizers that drive modern language models split words by corpus statistics, fragmenting semantically loaded suffixes and -- in the case of WordPiece and rule-based analyzers -- failing to decode their output back to the original text. This paper presents Morpheus, a neural morpheme-boundary model for Turkish that is at once a lossless, morphology-aware tokenizer and a word-embedding producer. A differentiable Poisson-binomial dynamic program turns per-character boundary probabilities into soft morpheme memberships during training and exact segments at inference, with no string normalization, so decode(encode(w)) = w holds by construction. Because the model is neural, the same forward pass that tokenizes also emits a structured word embedding. Among reversible tokenizers -- the only ones valid for generation -- Morpheus attains the lowest bits-per-character (1.425), roughly doubles the gold morphological alignment of the subword family (MorphScore macro-F1 0.61 vs. ~0.32), and uses ~19% less GPU memory than 64K-vocabulary subword tokenizers. As an embedder, frozen Morpheus vectors lead on lexical retrieval (root-family MAP 0.85) and same-root verification (ROC-AUC 1.00), surpassing the multilingual retriever BGE-M3 and BERTurk; on context- and inflection-dependent tasks (NER, case/number probing) the heavier contextual encoders remain ahead -- a trade-off we attribute to Morpheus's root-centric geometry. Code: https://github.com/lonewolf-rd/TurkishMorpheus; model: https://huggingface.co/lonewolflab/Morpheus-TR-50K; interactive demo: https://huggingface.co/spaces/lonewolflab/morpheus-tr-demo.