# From Tokens to Words: On the Inner Lexicon of LLMs

arXiv: https://arxiv.org/abs/2410.05864 · HTML: https://mlsanity.com/paper/2410.05864/

## Key facts

- Cited by 48 papers in the map (data through 14 Oct 2026).
- Builds on 53 of its 69 references that are in the map.
- First posted on arXiv 8 Oct 2024.

## Builds on (most cited first, 30 of 53)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Pointer Sentinel Mixture Models](https://mlsanity.com/paper/1609.07843/) (2016)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [Locating and Editing Factual Associations in GPT](https://mlsanity.com/paper/2202.05262/) (2022)
- [Don't Stop Pretraining: Adapt Language Models to Domains and Tasks](https://mlsanity.com/paper/2004.10964/) (2020)
- [Root Mean Square Layer Normalization](https://mlsanity.com/paper/1910.07467/) (2019)
- [Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation](https://mlsanity.com/paper/1609.08144/) (2016)
- [Transformer Feed-Forward Layers Are Key-Value Memories](https://mlsanity.com/paper/2012.14913/) (2020)
- [BERT Rediscovers the Classical NLP Pipeline](https://mlsanity.com/paper/1905.05950/) (2019)
- [Discovering Latent Knowledge in Language Models Without Supervision](https://mlsanity.com/paper/2212.03827/) (2022)
- [Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidates](https://mlsanity.com/paper/1804.10959/) (2018)
- [Eliciting Latent Predictions from Transformers with the Tuned Lens](https://mlsanity.com/paper/2303.08112/) (2023)
- [Knowledge Neurons in Pretrained Transformers](https://mlsanity.com/paper/2104.08696/) (2021)
- [Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Space](https://mlsanity.com/paper/2203.14680/) (2022)
- [Dissecting Recall of Factual Associations in Auto-Regressive Language Models](https://mlsanity.com/paper/2304.14767/) (2023)
- [What you can cram into a single vector: Probing sentence embeddings for linguistic properties](https://mlsanity.com/paper/1805.01070/) (2018)
- [Towards Best Practices of Activation Patching in Language Models: Metrics and Methods](https://mlsanity.com/paper/2309.16042/) (2023)
- [Finding Neurons in a Haystack: Case Studies with Sparse Probing](https://mlsanity.com/paper/2305.01610/) (2023)
- [Benchmarking Retrieval-Augmented Generation for Medicine](https://mlsanity.com/paper/2402.13178/) (2024)
- [Function Vectors in Large Language Models](https://mlsanity.com/paper/2310.15213/) (2023)
- [Patchscopes: A Unifying Framework for Inspecting Hidden Representations of Language Models](https://mlsanity.com/paper/2401.06102/) (2024)
- [Language Model Tokenizers Introduce Unfairness Between Languages](https://mlsanity.com/paper/2305.15425/) (2023)
- [Continual Pre-training of Language Models](https://mlsanity.com/paper/2302.03241/) (2023)
- [BPE-Dropout: Simple and Effective Subword Regularization](https://mlsanity.com/paper/1910.13267/) (2019)
- [Byte Pair Encoding is Suboptimal for Language Model Pretraining](https://mlsanity.com/paper/2004.03720/) (2020)
- [Do All Languages Cost the Same? Tokenization in the Era of Commercial Language Models](https://mlsanity.com/paper/2305.13707/) (2023)
- [The Geometry of Categorical and Hierarchical Concepts in Large Language Models](https://mlsanity.com/paper/2406.01506/) (2024)
- [Probing Pretrained Language Models for Lexical Semantics](https://mlsanity.com/paper/2010.05731/) (2020)
- [Adaptively Sparse Transformers](https://mlsanity.com/paper/1909.00015/) (2019)

## Built on by (most cited first, 30 of 48)

- [Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach](https://mlsanity.com/paper/2502.05171/) (2025)
- [SuperBPE: Space Travel for Language Models](https://mlsanity.com/paper/2503.13423/) (2025)
- [The Dual-Route Model of Induction](https://mlsanity.com/paper/2504.03022/) (2025)
- [Broken Tokens? Your Language Model can Secretly Handle Non-Canonical Tokenizations](https://mlsanity.com/paper/2506.19004/) (2025)
- [Probing Semantic Routing in Large Mixture-of-Expert Models](https://mlsanity.com/paper/2502.10928/) (2025)
- [Is the Reversal Curse a Binding Problem? Uncovering Limitations of Transformers from a Basic Generalization Failure](https://mlsanity.com/paper/2504.01928/) (2025)
- [Layers at Similar Depths Generate Similar Activations Across LLM Architectures](https://mlsanity.com/paper/2504.08775/) (2025)
- [Adversarial Tokenization](https://mlsanity.com/paper/2503.02174/) (2025)
- [The Strawberry Problem: Emergence of Character-level Understanding in Tokenized Language Models](https://mlsanity.com/paper/2505.14172/) (2025)
- [Measuring and Guiding Monosemanticity](https://mlsanity.com/paper/2506.19382/) (2025)
- [Learning to Skip the Middle Layers of Transformers](https://mlsanity.com/paper/2506.21103/) (2025)
- [Elevating Visual Perception in Multimodal LLMs with Visual Embedding Distillation](https://mlsanity.com/paper/2412.09585/) (2024)
- [Latent Reasoning with Supervised Thinking States](https://mlsanity.com/paper/2602.08332/) (2026)
- [AbbIE: Autoregressive Block-Based Iterative Encoder for Efficient Sequence Modeling](https://mlsanity.com/paper/2507.08567/) (2025)
- [Efficient Parallel Samplers for Recurrent-Depth Models and Their Connection to Diffusion Language Models](https://mlsanity.com/paper/2510.14961/) (2025)
- [Why Fine-Tuning Encourages Hallucinations and How to Fix It](https://mlsanity.com/paper/2604.15574/) (2026)
- [Understanding the Anchoring Effect of LLM with Synthetic Data: Existence, Mechanism, and Potential Mitigations](https://mlsanity.com/paper/2505.15392/) (2025)
- [Token Distillation: Attention-aware Input Embeddings For New Tokens](https://mlsanity.com/paper/2505.20133/) (2025)
- [Born a Transformer -- Always a Transformer? On the Effect of Pretraining on Architectural Abilities](https://mlsanity.com/paper/2505.21785/) (2025)
- [Model-Aware Tokenizer Transfer](https://mlsanity.com/paper/2510.21954/) (2025)
- [Weight-based Analysis of Detokenization in Language Models: Understanding the First Stage of Inference Without Inference](https://mlsanity.com/paper/2501.15754/) (2025)
- [Model Internal Sleuthing: Finding Lexical Identity and Inflectional Features in Modern Language Models](https://mlsanity.com/paper/2506.02132/) (2025)
- [Follow the Flow: On Information Flow Across Textual Tokens in Text-to-Image Models](https://mlsanity.com/paper/2504.01137/) (2025)
- [Spelling-out is not Straightforward: LLMs' Capability of Tokenization from Token to Characters](https://mlsanity.com/paper/2506.10641/) (2025)
- [Meeseeks: A Feedback-Driven, Iterative Self-Correction Benchmark evaluating LLMs' Instruction Following Capability](https://mlsanity.com/paper/2504.21625/) (2025)
- [Word Recovery in Large Language Models Enables Character-Level Tokenization Robustness](https://mlsanity.com/paper/2603.10771/) (2026)
- [Defragmenting Language Models: An Interpretability-based Approach for Vocabulary Expansion](https://mlsanity.com/paper/2604.16656/) (2026)
- [Shorthand for Thought: Compressing LLM Reasoning via Entropy-Guided Supertokens](https://mlsanity.com/paper/2604.26355/) (2026)
- [Enhancing LLM Character-Level Manipulation via Divide and Conquer](https://mlsanity.com/paper/2502.08180/) (2025)
- [Length Representations in Large Language Models](https://mlsanity.com/paper/2507.20398/) (2025)

## Abstract (arXiv)

Natural language is composed of words, but modern large language models (LLMs) process sub-words as input. A natural question raised by this discrepancy is whether LLMs encode words internally, and if so how. We present evidence that LLMs engage in an intrinsic detokenization process, where sub-word sequences are combined into coherent whole-word representations at their last token. Our experiments show that this process primarily takes place within the early and middle layers of the model. We further demonstrate its robustness to arbitrary splits (e.g., "cats" to "ca" and "ts"), typos, and importantly-to out-of-vocabulary words: when feeding the last token internal representations of such words to the model as input, it can "understand" them as the complete word despite never seeing such representations as input during training. Our findings suggest that LLMs maintain a latent vocabulary beyond the tokenizer's scope. These insights provide a practical, finetuning-free application for expanding the vocabulary of pre-trained models. By enabling the addition of new vocabulary words, we reduce input length and inference iterations, which reduces both space and model latency, with little to no loss in model accuracy.