# Token Erasure as a Footprint of Implicit Vocabulary Items in LLMs

arXiv: https://arxiv.org/abs/2406.20086 · HTML: https://mlsanity.com/paper/2406.20086/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 19 references that are in the map.
- First posted on arXiv 28 Jun 2024.

## Builds on (most cited first, 11 of 11)

- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [Locating and Editing Factual Associations in GPT](https://mlsanity.com/paper/2202.05262/) (2022)
- [The Pile: An 800GB Dataset of Diverse Text for Language Modeling](https://mlsanity.com/paper/2101.00027/) (2020)
- [Dissecting Recall of Factual Associations in Auto-Regressive Language Models](https://mlsanity.com/paper/2304.14767/) (2023)
- [Finding Neurons in a Haystack: Case Studies with Sparse Probing](https://mlsanity.com/paper/2305.01610/) (2023)
- [Eliciting Latent Knowledge from Quirky Language Models](https://mlsanity.com/paper/2312.01037/) (2023)
- [Backward Lens: Projecting Language Model Gradients into the Vocabulary Space](https://mlsanity.com/paper/2402.12865/) (2024)
- [Tokenization Matters! Degrading Large Language Models through Challenging Their Tokenization](https://mlsanity.com/paper/2405.17067/) (2024)
- [Biomedical Language Models are Robust to Sub-optimal Tokenization](https://mlsanity.com/paper/2306.17649/) (2023)
- [Evaluating Subword Tokenization: Alien Subword Composition and OOV Generalization Challenge](https://mlsanity.com/paper/2404.13292/) (2024)

## Built on by (most cited first, 18 of 18)

- [From Tokens to Words: On the Inner Lexicon of LLMs](https://mlsanity.com/paper/2410.05864/) (2024)
- [Evaluating and Steering Modality Preferences in Multimodal Large Language Model](https://mlsanity.com/paper/2505.20977/) (2025)
- [The Dual-Route Model of Induction](https://mlsanity.com/paper/2504.03022/) (2025)
- [Broken Tokens? Your Language Model can Secretly Handle Non-Canonical Tokenizations](https://mlsanity.com/paper/2506.19004/) (2025)
- [Elucidating Mechanisms of Demographic Bias in LLMs for Healthcare](https://mlsanity.com/paper/2502.13319/) (2025)
- [Follow the Flow: On Information Flow Across Textual Tokens in Text-to-Image Models](https://mlsanity.com/paper/2504.01137/) (2025)
- [When Sinks Help or Hurt: Unified Framework for Attention Sink in Large Vision-Language Models](https://mlsanity.com/paper/2604.03316/) (2026)
- [CARMA: Enhanced Compositionality in LLMs via Advanced Regularisation and Mutual Information Alignment](https://mlsanity.com/paper/2502.11066/) (2025)
- [Steering Language Models in Multi-Token Generation: A Case Study on Tense and Aspect](https://mlsanity.com/paper/2509.12065/) (2025)
- [Vocabulary embeddings organize linguistic structure early in language model training](https://mlsanity.com/paper/2510.07613/) (2025)
- [Word Recovery in Large Language Models Enables Character-Level Tokenization Robustness](https://mlsanity.com/paper/2603.10771/) (2026)
- [Vocab Diet: Reshaping the Vocabulary of LLMs via Vector Arithmetic](https://mlsanity.com/paper/2510.17001/) (2025)
- [Friends and Grandmothers in Silico: Localizing Entity Cells in Language Models](https://mlsanity.com/paper/2604.01404/) (2026)
- [I Walk the Line: Examining the Role of Gestalt Continuity in Object Binding for Vision Transformers](https://mlsanity.com/paper/2604.09942/) (2026)
- [Stochasticity in Tokenisation Improves Robustness](https://mlsanity.com/paper/2604.16037/) (2026)
- [In-Context Learning Without Copying](https://mlsanity.com/paper/2511.05743/) (2025)
- [Inside the LLM Word Factory](https://mlsanity.com/paper/2606.08562/) (2026)
- [Extracting Persona Subspaces Through Iterative Nullspace Projection For Modulation](https://mlsanity.com/paper/2610.04676/) (2026)

## Abstract (arXiv)

LLMs process text as sequences of tokens that roughly correspond to words, where less common words are represented by multiple tokens. However, individual tokens are often semantically unrelated to the meanings of the words/concepts they comprise. For example, Llama-2-7b's tokenizer splits the word "northeastern" into the tokens ['_n', 'ort', 'he', 'astern'], none of which correspond to semantically meaningful units like "north" or "east." Similarly, the overall meanings of named entities like "Neil Young" and multi-word expressions like "break a leg" cannot be directly inferred from their constituent tokens. Mechanistically, how do LLMs convert such arbitrary groups of tokens into useful higher-level representations? In this work, we find that last token representations of named entities and multi-token words exhibit a pronounced "erasure" effect, where information about previous and current tokens is rapidly forgotten in early layers. Using this observation, we propose a method to "read out" the implicit vocabulary of an autoregressive LLM by examining differences in token representations across layers, and present results of this method for Llama-2-7b and Llama-3-8B. To our knowledge, this is the first attempt to probe the implicit vocabulary of an LLM.