mlsanityData to 14 Oct 2026

Interpretable Next-token Prediction via the Generalized Induction Head

arXiv 2411.00066 ↗First posted 31 Oct 2024cs.CL cs.AI cs.LGcited by 3

Not on the map. Why?

Abstract (arXiv)

While large transformer models excel in predictive performance, their lack of interpretability restricts their usefulness in high-stakes domains. To remedy this, we propose the Generalized Induction-Head Model (GIM), an interpretable model for next-token prediction inspired by the observation of "induction heads" in LLMs. GIM is a retrieval-based module that identifies similar sequences in the input context by combining exact n-gram matching and fuzzy matching based on a neural similarity metric. We evaluate GIM in two settings: language modeling and fMRI response prediction. In language modeling, GIM improves next-token prediction by up to 25%p over interpretable baselines, significantly narrowing the gap with black-box LLMs. In an fMRI setting, GIM improves neural response prediction by 20% and offers insights into the language selectivity of the brain. GIM represents a significant step toward uniting interpretability and performance across domains. The code is available at https://github.com/ejkim47/generalized-induction-head.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 44 of its 95 references that are in the map. First posted on arXiv 31 Oct 2024.

Builds on

44 of 95 references.

References · 44

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  4. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  5. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,411
  6. 2016Concrete Problems in AI Safetycited by 1,642
  7. 2021Improving language models by retrieving from trillions of tokenscited by 1,293
  8. 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,165
  9. 2024The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scalecited by 1,160
  10. 2018Sanity Checks for Saliency Mapscited by 1,156
  11. 2018Self-Attention with Relative Position Representationscited by 1,120
  12. 2016Bag of Tricks for Efficient Text Classificationcited by 876
32 more
  1. 2019Generalization through Memorization: Nearest Neighbor Language Modelscited by 800
  2. 2019Temporal Fusion Transformers for Interpretable Multi-horizon Time Series Forecastingcited by 761
  3. 2020Artificial Intelligence, Values and Alignmentcited by 448
  4. 2021Interpretable Machine Learning: Fundamental Principles and 10 Grand Challengescited by 367
  5. 2016European Union regulations on algorithmic decision-making and a "right to explanation"cited by 344
  6. 2022Language in a Bottle: Language Model Guided Concept Bottlenecks for Interpretable Image Classificationcited by 312
  7. 2023Can Large Language Models Transform Computational Social Science?cited by 250
  8. 2024Retrieval Head Mechanistically Explains Long-Context Factualitycited by 232
  9. 2024Infini-gram: Scaling Unbounded n-gram Language Models to a Trillion Tokenscited by 153
  10. 2019Interpreting and improving natural-language processing (in machines) with natural language-processing (in the brain)cited by 128
  11. 2024A Practical Review of Mechanistic Interpretability for Transformer-Based Language Modelscited by 118
  12. 2022Pile of Law: Learning Responsible Data Filtering from the Law and a 256GB Open-Source Legal Datasetcited by 113
  13. 2024Rethinking Interpretability in the Era of Large Language Modelscited by 111
  14. 2023Large Language Models for Automated Open-domain Scientific Hypotheses Discoverycited by 100
  15. 2023Tell Your Model Where to Attend: Post-hoc Attention Steering for LLMscited by 91
  16. 2024Zero Shot Health Trajectory Prediction Using Transformercited by 77
  17. 2023Scaling laws for language encoding models in fMRIcited by 68
  18. 2023Explaining black box text modules in natural language with language modelscited by 62
  19. 2022Describing Differences between Text Distributions with Natural Languagecited by 54
  20. 2023Describing Differences in Image Sets with Natural Languagecited by 53
  21. 2021Disentangling Syntax and Semantics in the Brain with Deep Networkscited by 40
  22. 2022Augmenting Interpretable Models with LLMs during Trainingcited by 38
  23. 2022Joint processing of linguistic properties in brains and language modelscited by 37
  24. 2021Adaptive wavelet distillation from neural networks through interpretationscited by 32
  25. 2024Iteration Head: A Mechanistic Study of Chain-of-Thoughtcited by 31
  26. 2023Tree Prompting: Efficient Task Adaptation without Fine-Tuningcited by 18
  27. 2024Bayesian Concept Bottleneck Models with LLM Priorscited by 17
  28. 2023CHiLL: Zero-shot Custom Interpretable Feature Extraction from Clinical Notes with Large Language Modelscited by 15
  29. 2023Information-Restricted Neural Language Models Reveal Different Brain Regions' Sensitivity to Semantics, Syntax and Contextcited by 13
  30. 2023Humans and language models diverge when predicting repeating textcited by 13
  31. 2024Assessing Episodic Memory in LLMs with Sequence Order Recall Taskscited by 10
  32. 2024Great Memory, Shallow Reasoning: Limits of kNN-LMscited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 2.

3 papers.

  1. 2026Interpreting and Steering State-Space Models via Activation Subspace Bottleneckscited by 3
  2. 2026Prototype Language Modelscited by 2
  3. 2025Rethinking Reasoning in LLMs: Neuro-Symbolic Local RetoMaton Beyond ICL and CoTcited by 0