# Fishing for Magikarp: Automatically Detecting Under-trained Tokens in Large Language Models

arXiv: https://arxiv.org/abs/2405.05417 · HTML: https://mlsanity.com/paper/2405.05417/

## Key facts

- Cited by 46 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 39 references that are in the map.
- First posted on arXiv 8 May 2024.

## Builds on (most cited first, 20 of 20)

- [Qwen Technical Report](https://mlsanity.com/paper/2309.16609/) (2023)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [HuggingFace's Transformers: State-of-the-art Natural Language Processing](https://mlsanity.com/paper/1910.03771/) (2019)
- [Yi: Open Foundation Models by 01.AI](https://mlsanity.com/paper/2403.04652/) (2024)
- [GPT-NeoX-20B: An Open-Source Autoregressive Language Model](https://mlsanity.com/paper/2204.06745/) (2022)
- [Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidates](https://mlsanity.com/paper/1804.10959/) (2018)
- [OLMo: Accelerating the Science of Language Models](https://mlsanity.com/paper/2402.00838/) (2024)
- [ByT5: Towards a token-free future with pre-trained byte-to-byte models](https://mlsanity.com/paper/2105.13626/) (2021)
- [StarCoder 2 and The Stack v2: The Next Generation](https://mlsanity.com/paper/2402.19173/) (2024)
- [Jamba: A Hybrid Transformer-Mamba Language Model](https://mlsanity.com/paper/2403.19887/) (2024)
- [BPE-Dropout: Simple and Effective Subword Regularization](https://mlsanity.com/paper/1910.13267/) (2019)
- [Byte Pair Encoding is Suboptimal for Language Model Pretraining](https://mlsanity.com/paper/2004.03720/) (2020)
- [MEGABYTE: Predicting Million-byte Sequences with Multiscale Transformers](https://mlsanity.com/paper/2305.07185/) (2023)
- [Between words and characters: A Brief History of Open-Vocabulary Modeling and Tokenization in NLP](https://mlsanity.com/paper/2112.10508/) (2021)
- [Tying Word Vectors and Word Classifiers: A Loss Framework for Language Modeling](https://mlsanity.com/paper/1611.01462/) (2016)
- [Stable LM 2 1.6B Technical Report](https://mlsanity.com/paper/2402.17834/) (2024)
- [Coercing LLMs to do and reveal (almost) anything](https://mlsanity.com/paper/2402.14020/) (2024)
- [SpaceByte: Towards Deleting Tokenization from Large Language Modeling](https://mlsanity.com/paper/2404.14408/) (2024)
- [RakutenAI-7B: Extending Large Language Models for Japanese](https://mlsanity.com/paper/2403.15484/) (2024)
- [A Systematic Analysis of Vocabulary and BPE Settings for Optimal Fine-tuning of NMT: A Case Study of In-domain Translation](https://mlsanity.com/paper/2303.00722/) (2023)

## Built on by (most cited first, 30 of 46)

- [SuperBPE: Space Travel for Language Models](https://mlsanity.com/paper/2503.13423/) (2025)
- [LLMmap: Fingerprinting For Large Language Models](https://mlsanity.com/paper/2407.15847/) (2024)
- [Shared Global and Local Geometry of Language Model Embeddings](https://mlsanity.com/paper/2503.21073/) (2025)
- [Tokenization Matters! Degrading Large Language Models through Challenging Their Tokenization](https://mlsanity.com/paper/2405.17067/) (2024)
- [Zero-Shot Tokenizer Transfer](https://mlsanity.com/paper/2405.07883/) (2024)
- [Mind the Inconspicuous: Revealing the Hidden Weakness in Aligned LLMs' Refusal Boundaries](https://mlsanity.com/paper/2405.20653/) (2024)
- [The Dual-Route Model of Induction](https://mlsanity.com/paper/2504.03022/) (2025)
- [Special Characters Attack: Toward Scalable Training Data Extraction From Large Language Models](https://mlsanity.com/paper/2405.05990/) (2024)
- [Dense SAE Latents Are Features, Not Bugs](https://mlsanity.com/paper/2506.15679/) (2025)
- [Dual Process Learning: Controlling Use of In-Context vs. In-Weights Strategies with Weight Forgetting](https://mlsanity.com/paper/2406.00053/) (2024)
- [Data Mixture Inference: What do BPE Tokenizers Reveal about their Training Data?](https://mlsanity.com/paper/2407.16607/) (2024)
- [One Tokenizer To Rule Them All: Emergent Language Plasticity via Multilingual Tokenizers](https://mlsanity.com/paper/2506.10766/) (2025)
- [Bolmo: Byteifying the Next Generation of Language Models](https://mlsanity.com/paper/2512.15586/) (2025)
- [Problematic Tokens: Tokenizer Bias in Large Language Models](https://mlsanity.com/paper/2406.11214/) (2024)
- [Understanding Likelihood Over-optimisation in Direct Alignment Algorithms](https://mlsanity.com/paper/2410.11677/) (2024)
- [Exploiting Vocabulary Frequency Imbalance in Language Model Pre-training](https://mlsanity.com/paper/2508.15390/) (2025)
- [Tokenization for Molecular Foundation Models](https://mlsanity.com/paper/2409.15370/) (2024)
- [MinGram: A Minimalist Unigram Tokenizer with High Compression and Competitive Morphological Alignment](https://mlsanity.com/paper/2606.27019/) (2026)
- [GlitchMiner: Mining Glitch Tokens in Large Language Models via Gradient-based Discrete Optimization](https://mlsanity.com/paper/2410.15052/) (2024)
- [Probing the topology of the space of tokens with structured prompts](https://mlsanity.com/paper/2503.15421/) (2025)
- [From Macro to Micro: Probing Dataset Diversity in Language Model Fine-Tuning](https://mlsanity.com/paper/2505.24768/) (2025)
- [FPEdit: Robust LLM Fingerprinting through Localized Parameter Editing](https://mlsanity.com/paper/2508.02092/) (2025)
- [UTF-8 Plumbing: Byte-level Tokenizers Unavoidably Enable LLMs to Generate Ill-formed UTF-8](https://mlsanity.com/paper/2511.05578/) (2025)
- [Proxy Compression for Language Modeling](https://mlsanity.com/paper/2602.04289/) (2026)
- [LiteToken: Removing Intermediate Merge Residues From BPE Tokenizers](https://mlsanity.com/paper/2602.04706/) (2026)
- [Disentangling MLP Neuron Weights in Vocabulary Space](https://mlsanity.com/paper/2604.06005/) (2026)
- [Qtok: A Comprehensive Framework for Evaluating Multilingual Tokenizer Quality in Large Language Models](https://mlsanity.com/paper/2410.12989/) (2024)
- [Vocabulary embeddings organize linguistic structure early in language model training](https://mlsanity.com/paper/2510.07613/) (2025)
- [Shorthand for Thought: Compressing LLM Reasoning via Entropy-Guided Supertokens](https://mlsanity.com/paper/2604.26355/) (2026)
- [Decoupling the Benefits of Subword Tokenization for Language Model Training via Byte-level Simulation](https://mlsanity.com/paper/2604.27263/) (2026)

## Abstract (arXiv)

The disconnect between tokenizer creation and model training in language models allows for specific inputs, such as the infamous SolidGoldMagikarp token, to induce unwanted model behaviour. Although such `glitch tokens', tokens present in the tokenizer vocabulary but that are nearly or entirely absent during model training, have been observed across various models, a reliable method to identify and address them has been missing. We present a comprehensive analysis of Large Language Model tokenizers, specifically targeting this issue of detecting under-trained tokens. Through a combination of tokenizer analysis, model weight-based indicators, and prompting techniques, we develop novel and effective methods for automatically detecting these problematic tokens. Our findings demonstrate the prevalence of such tokens across a diverse set of models and provide insights into improving the efficiency and safety of language models.