mlsanityData to 14 Oct 2026

Information Flow Routes: Automatically Interpreting Language Models at Scale

arXiv 2403.00824 ↗First posted 27 Feb 2024cs.CL cs.AIcited by 95

Not on the map. Why?

Cited by 24 in Mechanistic interpretability, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Information flows by routes inside the network via mechanisms implemented in the model. These routes can be represented as graphs where nodes correspond to token representations and edges to operations inside the network. We automatically build these graphs in a top-down manner, for each prediction leaving only the most important nodes and edges. In contrast to the existing workflows relying on activation patching, we do this through attribution: this allows us to efficiently uncover existing circuits with just a single forward pass. Additionally, the applicability of our method is far beyond patching: we do not need a human to carefully design prediction templates, and we can extract information flow routes for any prediction (not just the ones among the allowed templates). As a result, we can talk about model behavior in general, for specific types of predictions, or different domains. We experiment with Llama 2 and show that the role of some attention heads is overall important, e.g. previous token heads and subword merging heads. Next, we find similarities in Llama 2 behavior when handling tokens of the same part of speech. Finally, we show that some model components can be specialized on domains such as coding or multilingual texts.

Key facts. Cited by 95 papers in the map (data through 14 Oct 2026). Builds on 26 of its 42 references that are in the map. First posted on arXiv 27 Feb 2024.

Builds on

26 of 42 references.

References · 26

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  5. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  6. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  7. 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,336
  8. 2022No Language Left Behind: Scaling Human-Centered Machine Translationcited by 1,312
  9. 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,165
  10. 2019Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Prunedcited by 856
  11. 2023Towards Automated Circuit Discovery for Mechanistic Interpretabilitycited by 724
  12. 2021Knowledge Neurons in Pretrained Transformerscited by 601
14 more
  1. 2021The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translationcited by 560
  2. 2023In-Context Learning Creates Task Vectorscited by 312
  3. 2023Attribution Patching Outperforms Automated Circuit Discoverycited by 209
  4. 2023How does GPT-2 compute greater-than?: Interpreting mathematical abilities in a pre-trained language modelcited by 202
  5. 2023The Hydra Effect: Emergent Self-repair in Language Model Computationscited by 156
  6. 2019Adaptively Sparse Transformerscited by 155
  7. 2023Hallucinations in Large Multilingual Translation Modelscited by 129
  8. 2020The elephant in the interpretability room: Why use attention as explanation when we have saliency methods?cited by 122
  9. 2018Identifying and Controlling Important Neurons in Neural Machine Translationcited by 111
  10. 2022Detecting and Mitigating Hallucinations in Machine Translation: Model Internal Workings Alone Do Well, Sentence Similarity Even Bettercited by 75
  11. 2022Measuring the Mixing of Contextual Information in the Transformercited by 65
  12. 2024Spectral Filters, Dark Signals, and Attention Sinkscited by 56
  13. 2023HalOmi: A Manually Annotated Benchmark for Multilingual Hallucination and Omission Detection in Machine Translationcited by 35
  14. 2023Traveling Words: A Geometric Interpretation of Transformerscited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 29; 2025: 39; 2026: 27.

95 papers.

Sort:

By area:

  1. 2024Mechanistic Interpretability for AI Safety -- A Reviewcited by 422
  2. 2024The Remarkable Robustness of LLMs: Stages of Inference?cited by 146
  3. 2025See What You Are Told: Visual Attention Sink in Large Multimodal Modelscited by 132
  4. 2024ReDeEP: Detecting Hallucination in Retrieval-Augmented Generation via Mechanistic Interpretabilitycited by 101
  5. 2024A Primer on the Inner Workings of Transformer-based Language Modelscited by 85
  6. 2024Attention Heads of Large Language Models: A Surveycited by 67
  7. 2024LLM Circuit Analyses Are Consistent Across Training and Scalecited by 63
  8. 2024Knowledge Circuits in Pretrained Transformerscited by 60
  9. 2025MIB: A Mechanistic Interpretability Benchmarkcited by 56
  10. 2024From Tokens to Words: On the Inner Lexicon of LLMscited by 48
  11. 2026When Models Manipulate Manifolds: The Geometry of a Counting Taskcited by 46
  12. 2024Activation Scaling for Steering and Interpreting Language Modelscited by 35
38 more
  1. 2024Unveiling Visual Perception in Language Models: An Attention Head Analysis Approachcited by 35
  2. 2025Detection and Mitigation of Hallucination in Large Reasoning Models: A Mechanistic Perspectivecited by 34
  3. 2024The Same But Different: Structural Similarities and Differences in Multilingual Language Modelingcited by 28
  4. 2026Locate, Steer, and Improve: A Practical Survey of Actionable Mechanistic Interpretability in Large Language Modelscited by 27
  5. 2024Mechanistic?cited by 25
  6. 2025The Dual-Route Model of Inductioncited by 24
  7. 2024Missed Causes and Ambiguous Effects: Counterfactuals Pose Challenges for Interpreting Neural Networkscited by 23
  8. 2025How Do LLMs Acquire New Knowledge? A Knowledge Circuits Perspective on Continual Pre-Trainingcited by 23
  9. 2024Editing Conceptual Knowledge for Large Language Modelscited by 21
  10. 2024LRP4RAG: Detecting Hallucinations in Retrieval-Augmented Generation via Layer-wise Relevance Propagationcited by 17
  11. 2024LM Transparency Tool: Interactive Tool for Analyzing Transformer Language Modelscited by 16
  12. 2025LAVa: Layer-wise KV Cache Eviction with Dynamic Budget Allocationcited by 16
  13. 2025Does Time Have Its Place? Temporal Heads: Where Language Models Recall Time-specific Informationcited by 15
  14. 2025LLMs as a synthesis between symbolic and distributed approaches to languagecited by 13
  15. 2025RelP: Faithful and Efficient Circuit Discovery in Language Models via Relevance Patchingcited by 13
  16. 2024LLMs Faithfully and Iteratively Compute Answers During CoT: A Systematic Analysis With Multi-step Arithmeticscited by 11
  17. 2025A Close Look at Decomposition-based XAI-Methods for Transformer Language Modelscited by 11
  18. 2025The Atlas of In-Context Learning: How Attention Heads Shape In-Context Retrieval Augmentationcited by 11
  19. 2024Transformer Explainer: Learning LLM Transformers with Interactive Visual Explanation and Experimentationcited by 10
  20. 2025Contextualize-then-Aggregate: Circuits for In-Context Learning in Gemma-2 2Bcited by 10
  21. 2025Interpretation Meets Safety: A Survey on Interpretation Methods and Tools for Improving LLM Safetycited by 10
  22. 2024Unconditional Truthfulness: Learning Unconditional Uncertainty of Large Language Modelscited by 9
  23. 2025HyperDAS: Towards Automating Mechanistic Interpretability with Hypernetworkscited by 9
  24. 2026Conditional Co-Ablation: Recovering Self-Repair Backups in Transformer Circuitscited by 9
  25. 2024Transformer Normalisation Layers and the Independence of Semantic Subspacescited by 6
  26. 2024SPES: Spectrogram Perturbation for Explainable Speech-to-Text Generationcited by 6
  27. 2025Unboxing the Black Box: Mechanistic Interpretability for Algorithmic Understanding of Neural Networkscited by 6
  28. 2026Subliminal Learning is a LoRA Artifactcited by 6
  29. 2025Correcting Gradient-Based Circuit Localization via Interaction-Aware Backpropagationcited by 5
  30. 2025Capturing Gaze Shifts for Guidance: Cross-Modal Fusion Enhancement for VLM Hallucination Mitigationcited by 5
  31. 2026Measuring Affinity between Attention-Head Weight Subspaces via the Projection Kernelcited by 5
  32. 2024Sparse Attention Decomposition Applied to Circuit Tracingcited by 4
  33. 2024Jet Expansions of Residual Computationcited by 4
  34. 2025Explanations of Large Language Models Explain Language Representations in the Braincited by 4
  35. 2025Generalized Attention Flow: Feature Attribution for Transformer Models via Maximum Flowcited by 4
  36. 2025Unraveling Syntax: Language Modeling and the Substructure of Grammarscited by 4
  37. 2025SEReDeEP: Hallucination Detection in Retrieval-Augmented Models via Semantic Entropy and Context-Parameter Fusioncited by 3
  38. 2025Attribution-Guided Pruning for Insight and Control: Circuit Discovery and Targeted Correction in Small-scale LLMscited by 3

50 most cited of 95 All 95 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Analyze Feature Flow to Enhance Interpretation and Steering in Language Models
  2. 2025GraphGhost: Tracing Structures Behind Large Language Models
  3. 2026Atomic Information Flow: A Network Flow Model for Tool Attributions in RAG Systems
  4. 2026When Graph Tokens Sink: A Mechanistic Analysis of Graph Language Models
  5. 2024ToolNet: Connecting Large Language Models with Massive Tools via Tool Graph
  6. 2026ConvergeFlow: Language Flow with Provable Convergence to Token Embeddings
  7. 2026A Geometric Perspective on Next-Token Prediction in Large Language Models: Three Emerging Phases
  8. 2026How Does Reasoning Flow? Tracing Attention-Induced Information Flow for Targeted RL in LLMs