The Limits of BPE Tokenization in Polish: Segmentation-Flexional Forms, Grammatical Anchoring, and First-Person Stability in Inflectional Language Models
Not on the map. Why?
Abstract (arXiv)
This article analyzes BPE tokenization in Polish as a test case for the limits of statistical segmentation in an inflectional language. It asks whether frequency-based tokenization preserves linguistically relevant units, including orthographic form, phonemic and syllabic segmentation, derivational structure, inflectional endings, grammatical form, and the speaking subject. The material includes diagnostic words, a children's text, selected forms from the Preamble to the Constitution of the Republic of Poland, word-family tests, and examples with Polish diacritics and nasal vowels. BPE tokenizers may produce segments that coincide with syllabic or morphologically interpretable divisions, but remain dependent on the frequency of written forms. They do not systematically map orthographic representation onto phonemic structure or context-dependent phonetic realization. The results show that BPE stabilizes frequent surface fragments of grammatical exponents rather than grammatical categories themselves. A form such as ustanawiamy is not merely a sequence ending in -y, but a verbal form anchored in conjugation, person, number, tense, mood, and aspect. The article develops the concept of grammatical form anchoring. In Polish, forms such as poszlam, zrobilam, or bylam can establish the position of the speaking subject without an explicit pronoun. In interaction with AI, this exposes a further problem: a language model does not possess a stable grammatical "I", but reconstructs it contextually and may mirror the user's forms or shift grammatical gender. Roclawski's segmentation-flexional forms are proposed as a diagnostic framework for evaluating tokenization boundaries. More stable modeling of Polish may require sublexical stabilization, anchoring grammatical form in the inflectional system, representing sentence patterns and verbal valency, and maintaining the grammatical "I" in dialogue.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 21 of its 57 references that are in the map. First posted on arXiv 17 Jul 2026.
Builds on
21 of 57 references.
References · 21
- 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,918
- 2020Long Range Arena: A Benchmark for Efficient Transformerscited by 700
- 2024Do Llamas Work in English? On the Latent Language of Multilingual Transformerscited by 281
- 2020Universal Dependencies v2: An Evergrowing Multilingual Treebank Collectioncited by 271
- 2020Byte Pair Encoding is Suboptimal for Language Model Pretrainingcited by 192
- 2023Do All Languages Cost the Same? Tokenization in the Era of Commercial Language Modelscited by 192
- 2023Tokenizer Choice For LLM Training: Negligible or Crucial?cited by 108
- 2024Why do language models perform worse for morphologically complex languages?cited by 62
- 2024Language Imbalance Driven Rewarding for Multilingual Self-improvingcited by 28
- 2025MorphBPE: A Morpho-Aware Tokenizer Bridging Linguistic Complexity for Efficient LLM Training Across Morphologiescited by 25
- 2025Evaluating Morphological Alignment of Tokenizers in 70 Languagescited by 22
- 2025One ruler to measure them all: Benchmarking multilingual long-context language modelscited by 19
9 more
- 2024Tokenization and Morphology in Multilingual Language Models: A Comparative Analysis of mT5 and ByT5cited by 14
- 2025The Translation Barrier Hypothesis: Multilingual Generation with Large Language Models Suffers from Implicit Translation Failurecited by 10
- 2025Rethinking Tokenization for Rich Morphology: The Dominance of Unigram over BPE and Morphological Alignmentcited by 9
- 2023Morphosyntactic probing of multilingual BERT modelscited by 8
- 2025PLLuM: A Family of Polish Large Language Modelscited by 8
- 2025Bielik v3 Small: Technical Reportcited by 5
- 2022Morphology Without Borders: Clause-Level Morphologycited by 2
- 2025Contextual morphologically-guided tokenization for Latin encoder modelscited by 1
- 2026Advancing Polish Language Modeling through Tokenizer Optimization in the Bielik v3 7B and 11B Seriescited by 1
Built on by
Not cited yet.