mlsanityData to 7 Oct 2026

Fishing for Magikarp: Automatically Detecting Under-trained Tokens in Large Language Models

arXiv 2405.05417 ↗First posted 8 May 2024cs.CLcited by 45

Not on the map. Why?

Abstract (arXiv)

The disconnect between tokenizer creation and model training in language models allows for specific inputs, such as the infamous SolidGoldMagikarp token, to induce unwanted model behaviour. Although such `glitch tokens', tokens present in the tokenizer vocabulary but that are nearly or entirely absent during model training, have been observed across various models, a reliable method to identify and address them has been missing. We present a comprehensive analysis of Large Language Model tokenizers, specifically targeting this issue of detecting under-trained tokens. Through a combination of tokenizer analysis, model weight-based indicators, and prompting techniques, we develop novel and effective methods for automatically detecting these problematic tokens. Our findings demonstrate the prevalence of such tokens across a diverse set of models and provide insights into improving the efficiency and safety of language models.

Key facts. Cited by 45 papers in the map (data through 7 Oct 2026). Builds on 20 of its 39 references that are in the map. First posted on arXiv 8 May 2024.

Builds on

20 of 39 references.

References · 20

  1. 2023Qwen Technical Reportcited by 3,273
  2. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
  3. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,201
  4. 2022GPT-NeoX-20B: An Open-Source Autoregressive Language Modelcited by 711
  5. 2024Yi: Open Foundation Models by 01.AIcited by 706
  6. 2018Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidatescited by 632
  7. 2024OLMo: Accelerating the Science of Language Modelscited by 599
  8. 2021ByT5: Towards a token-free future with pre-trained byte-to-byte modelscited by 504
  9. 2024StarCoder 2 and The Stack v2: The Next Generationcited by 484
  10. 2024Jamba: A Hybrid Transformer-Mamba Language Modelcited by 390
  11. 2019BPE-Dropout: Simple and Effective Subword Regularizationcited by 189
  12. 2020Byte Pair Encoding is Suboptimal for Language Model Pretrainingcited by 189
8 more
  1. 2023MEGABYTE: Predicting Million-byte Sequences with Multiscale Transformerscited by 164
  2. 2021Between words and characters: A Brief History of Open-Vocabulary Modeling and Tokenization in NLPcited by 130
  3. 2024Coercing LLMs to do and reveal (almost) anythingcited by 88
  4. 2016Tying Word Vectors and Word Classifiers: A Loss Framework for Language Modelingcited by 87
  5. 2024Stable LM 2 1.6B Technical Reportcited by 86
  6. 2024SpaceByte: Towards Deleting Tokenization from Large Language Modelingcited by 31
  7. 2024RakutenAI-7B: Extending Large Language Models for Japanesecited by 13
  8. 2023A Systematic Analysis of Vocabulary and BPE Settings for Optimal Fine-tuning of NMT: A Case Study of In-domain Translationcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 13; 2025: 15; 2026: 17.

45 papers.

Sort:

By area:

  1. 2025SuperBPE: Space Travel for Language Modelscited by 58
  2. 2024LLMmap: Fingerprinting For Large Language Modelscited by 56
  3. 2025Shared Global and Local Geometry of Language Model Embeddingscited by 34
  4. 2024Tokenization Matters! Degrading Large Language Models through Challenging Their Tokenizationcited by 32
  5. 2024Zero-Shot Tokenizer Transfercited by 30
  6. 2024Mind the Inconspicuous: Revealing the Hidden Weakness in Aligned LLMs' Refusal Boundariescited by 26
  7. 2025The Dual-Route Model of Inductioncited by 22
  8. 2025Dense SAE Latents Are Features, Not Bugscited by 17
  9. 2024Special Characters Attack: Toward Scalable Training Data Extraction From Large Language Modelscited by 16
  10. 2024Dual Process Learning: Controlling Use of In-Context vs. In-Weights Strategies with Weight Forgettingcited by 15
  11. 2024Data Mixture Inference: What do BPE Tokenizers Reveal about their Training Data?cited by 13
  12. 2025One Tokenizer To Rule Them All: Emergent Language Plasticity via Multilingual Tokenizerscited by 11
33 more
  1. 2024Problematic Tokens: Tokenizer Bias in Large Language Modelscited by 8
  2. 2025Bolmo: Byteifying the Next Generation of Language Modelscited by 8
  3. 2025Exploiting Vocabulary Frequency Imbalance in Language Model Pre-trainingcited by 7
  4. 2024Understanding Likelihood Over-optimisation in Direct Alignment Algorithmscited by 6
  5. 2026MinGram: A Minimalist Unigram Tokenizer with High Compression and Competitive Morphological Alignmentcited by 5
  6. 2024Tokenization for Molecular Foundation Modelscited by 4
  7. 2024GlitchMiner: Mining Glitch Tokens in Large Language Models via Gradient-based Discrete Optimizationcited by 3
  8. 2025Probing the topology of the space of tokens with structured promptscited by 3
  9. 2025From Macro to Micro: Probing Dataset Diversity in Language Model Fine-Tuningcited by 3
  10. 2025UTF-8 Plumbing: Byte-level Tokenizers Unavoidably Enable LLMs to Generate Ill-formed UTF-8cited by 3
  11. 2026Proxy Compression for Language Modelingcited by 3
  12. 2026LiteToken: Removing Intermediate Merge Residues From BPE Tokenizerscited by 3
  13. 2024Qtok: A Comprehensive Framework for Evaluating Multilingual Tokenizer Quality in Large Language Modelscited by 2
  14. 2025FPEdit: Robust LLM Fingerprinting through Localized Parameter Editingcited by 2
  15. 2026Shorthand for Thought: Compressing LLM Reasoning via Entropy-Guided Supertokenscited by 2
  16. 2026Decoupling the Benefits of Subword Tokenization for Language Model Training via Byte-level Simulationcited by 2
  17. 2025Alternatives To Next Token Prediction In Text Generation -- A Surveycited by 1
  18. 2025A Cartography of Open Collaboration in Open Source AI: Mapping Practices, Motivations, and Governance in 14 Open Large Language Model Projectscited by 1
  19. 2025Vocabulary embeddings organize linguistic structure early in language model trainingcited by 1
  20. 2026What Cosine Similarity of Label Representations Can and Cannot Tell uscited by 1
  21. 2026Disentangling MLP Neuron Weights in Vocabulary Spacecited by 1
  22. 2026MIXAR: Scaling Autoregressive Pixel-based Language Models to Multiple Languages and Scriptscited by 1
  23. 2026Scratchpad Patching: Decoupling Compute from Patch Size in Byte-Level Language Modelscited by 1
  24. 2026Writing-System-Level Tokenizer Adaptation for Byte-Level BPEcited by 1
  25. 2024Batching BPE Tokenization Mergescited by 0
  26. 2025`A la recherche du sens perdu: your favourite LLM might have more to say than you can understandcited by 0
  27. 2026Toten: A Knowledge-Based System For Structure-Preserving Representation Of Physical Quantities And Technical Notation In Brazilian Portuguesecited by 0
  28. 2026Where to cut, how deep: BPE and Unigram-LM on chemistry SMILEScited by 0
  29. 2026A Hub of Short Rows Inflates Intrinsic Dimension Estimation of Token Embeddingscited by 0
  30. 2026The Geometry of Ignorance: LLMs Know When to Temper Bayesian Priorscited by 0
  31. 2026Why Adaptive Optimizers Underestimate Rare Tokenscited by 0
  32. 2026Counting and Min-Cost Encoding for Tokenization in Large Language Modelscited by 0
  33. 2026HeadEdit: Calibrating Language Model Behavior Through the Frozen Unembedding Matrixcited by 0