mlsanityData to 7 Oct 2026

Hopping Too Late: Exploring the Limitations of Large Language Models on Multi-Hop Queries

arXiv 2406.12775 ↗First posted 18 Jun 2024cs.CLcited by 78

Not on the map. Why?

Cited by 39 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Large language models (LLMs) can solve complex multi-step problems, but little is known about how these computations are implemented internally. Motivated by this, we study how LLMs answer multi-hop queries such as "The spouse of the performer of Imagine is". These queries require two information extraction steps: a latent one for resolving the first hop ("the performer of Imagine") into the bridge entity (John Lennon), and another for resolving the second hop ("the spouse of John Lennon") into the target entity (Yoko Ono). Understanding how the latent step is computed internally is key to understanding the overall computation. By carefully analyzing the internal computations of transformer-based LLMs, we discover that the bridge entity is resolved in the early layers of the model. Then, only after this resolution, the two-hop query is solved in the later layers. Because the second hop commences in later layers, there could be cases where these layers no longer encode the necessary knowledge for correctly predicting the answer. Motivated by this, we propose a novel "back-patching" analysis method whereby a hidden representation from a later layer is patched back to an earlier layer. We find that in up to 66% of previously incorrect cases there exists a back-patch that results in the correct generation of the answer, showing that the later layers indeed sometimes lack the needed functionality. Overall, our methods and findings open further opportunities for understanding and improving latent reasoning in transformer-based LLMs.

Key facts. Cited by 78 papers in the map (data through 7 Oct 2026). Builds on 28 of its 43 references that are in the map. First posted on arXiv 18 Jun 2024.

Builds on

28 of 43 references.

References · 28

  1. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  2. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  3. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,734
  4. 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,307
  5. 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,130
  6. 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,054
  7. 2021Probing Classifiers: Promises, Shortcomings, and Advancescited by 831
  8. 2023Progress measures for grokking via mechanistic interpretabilitycited by 822
  9. 2023Towards Automated Circuit Discovery for Mechanistic Interpretabilitycited by 700
  10. 2021Editing Factual Knowledge in Language Modelscited by 548
  11. 2022Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Spacecited by 522
  12. 2023Dissecting Recall of Factual Associations in Auto-Regressive Language Modelscited by 512
16 more
  1. 2023Faith and Fate: Limits of Transformers on Compositionalitycited by 496
  2. 2023MQuAKE: Assessing Knowledge Editing in Language Models via Multi-Hop Questionscited by 292
  3. 2018Analysis Methods in Neural Language Processing: A Surveycited by 276
  4. 2023Evaluating the Ripple Effects of Knowledge Editing in Language Modelscited by 238
  5. 2024Patchscopes: A Unifying Framework for Inspecting Hidden Representations of Language Modelscited by 225
  6. 2024Do Large Language Models Latently Perform Multi-Hop Reasoning?cited by 165
  7. 2023The Hydra Effect: Emergent Self-repair in Language Model Computationscited by 145
  8. 2023Language Models Implement Simple Word2Vec-style Vector Arithmeticcited by 97
  9. 2024Grokked Transformers are Implicit Reasoners: A Mechanistic Journey to the Edge of Generalizationcited by 78
  10. 2023Can LMs Learn New Entities from Descriptions? Challenges in Propagating Injected Knowledgecited by 66
  11. 2023Towards a Mechanistic Interpretation of Multi-Step Reasoning Capabilities of Language Modelscited by 50
  12. 2024A Mechanistic Analysis of a Transformer Trained on a Symbolic Multi-Step Reasoning Taskcited by 49
  13. 2024Understanding and Patching Compositional Reasoning in LLMscited by 45
  14. 2023The Impact of Depth on Compositional Generalization in Transformer Language Modelscited by 37
  15. 2023Memory Injections: Correcting Multi-Hop Reasoning Failures during Inference in Transformer-Based Language Modelscited by 21
  16. 2022Does Your Model Classify Entities Reasonably? Diagnosing and Mitigating Spurious Correlations in Entity Typingcited by 16

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 14; 2025: 37; 2026: 27.

78 papers.

Sort:

By area:

  1. 2024Training Large Language Models to Reason in a Continuous Latent Spacecited by 763
  2. 2025A Survey of Efficient Reasoning for Large Reasoning Models: Language, Multimodality, and Beyondcited by 131
  3. 2025Machine Mental Imagery: Empower Multimodal Reasoning with Latent Visual Tokenscited by 120
  4. 2024A Practical Review of Mechanistic Interpretability for Transformer-Based Language Modelscited by 113
  5. 2025Soft Thinking: Unlocking the Reasoning Potential of LLMs in Continuous Concept Spacecited by 109
  6. 2025Efficient Reasoning with Hidden Thinkingcited by 72
  7. 2025A Survey on Latent Reasoningcited by 71
  8. 2025Think with 3D: Geometric Imagination Grounded Spatial Reasoning from Limited Viewscited by 58
  9. 2025PonderLM: Pretraining Language Models to Ponder in Continuous Spacecited by 44
  10. 2025Measuring Chain of Thought Faithfulness by Unlearning Reasoning Stepscited by 43
  11. 2025SAKURA: On the Multi-hop Reasoning of Large Audio-Language Models Based on Speech and Audio Informationcited by 42
  12. 2025Towards Holistic Evaluation of Large Audio-Language Models: A Comprehensive Surveycited by 38
38 more
  1. 2024Locate-then-edit for Multi-hop Factual Recall under Knowledge Editingcited by 36
  2. 2025Same Task, Different Circuits: Disentangling Modality-Specific Mechanisms in VLMscited by 35
  3. 2025Hybrid Latent Reasoning via Reinforcement Learningcited by 30
  4. 2024Do Large Language Models Perform Latent Multi-Hop Reasoning without Exploiting Shortcuts?cited by 26
  5. 2025Back Attention: Understanding and Enhancing Multi-Hop Reasoning in Large Language Modelscited by 23
  6. 2026Locate, Steer, and Improve: A Practical Survey of Actionable Mechanistic Interpretability in Large Language Modelscited by 22
  7. 2025The Reasoning-Memorization Interplay in Language Models Is Mediated by a Single Directioncited by 21
  8. 2025How Do LLMs Perform Two-Hop Reasoning in Context?cited by 20
  9. 2025AudioLens: A Closer Look at Auditory Attribute Perception of Large Audio-Language Modelscited by 18
  10. 2024MIRAGE: Evaluating and Explaining Inductive Reasoning Process in Language Modelscited by 17
  11. 2026Multiplex Thinking: Reasoning via Token-wise Branch-and-Mergecited by 16
  12. 2024Lessons from Studying Two-Hop Latent Reasoningcited by 15
  13. 2025Implicit Reasoning in Transformers is Reasoning through Shortcutscited by 14
  14. 2024Extractive Structures Learned in Pretraining Enable Generalization on Finetuned Factscited by 13
  15. 2024Performance Gap in Entity Knowledge Extraction Across Modalities in Vision Language Modelscited by 12
  16. 2024Racing Thoughts: Explaining Contextualization Errors in Large Language Modelscited by 11
  17. 2025Compositional Generalization from Learned Skills via CoT Training: A Theoretical and Structural Analysis for Reasoningcited by 10
  18. 2025Fundamental Limitations in Pointwise Defences of LLM Finetuning APIscited by 10
  19. 2025Generalization or Hallucination? Understanding Out-of-Context Reasoning in Transformerscited by 10
  20. 2026Do Latent-CoT Models Think Step-by-Step? A Mechanistic Study on Sequential Reasoning Taskscited by 10
  21. 2025CaKE: Circuit-aware Editing Enables Generalizable Knowledge Learnerscited by 9
  22. 2025Causality Matters: How Temporal Information Emerges in Video Language Modelscited by 9
  23. 2026Latent Reasoning with Supervised Thinking Statescited by 8
  24. 2024Defining Knowledge: Bridging Epistemology and Large Language Modelscited by 7
  25. 2025Can LLMs Reconcile Knowledge Conflicts in Counterfactual Reasoningcited by 7
  26. 2025AbbIE: Autoregressive Block-Based Iterative Encoder for Efficient Sequence Modelingcited by 7
  27. 2025How Do Language Models Compose Functions?cited by 7
  28. 2024Thinking with Knowledge Graphs: Enhancing LLM Reasoning Through Structured Datacited by 6
  29. 2025Too Late to Recall: Explaining the Two-Hop Problem in Multimodal Knowledge Retrievalcited by 6
  30. 2025Optimizing Multi-Hop Document Retrieval Through Intermediate Representationscited by 5
  31. 2025All for One: LLMs Solve Mental Math at the Last Token With Information Transferred From Other Tokenscited by 5
  32. 2024Fact Recall, Heuristics or Pure Guesswork? Precise Interpretations of Language Models for Fact Completioncited by 4
  33. 2024Benchmarking and Understanding Compositional Relational Reasoning of LLMscited by 4
  34. 2025Reasoning Can Hurt the Inductive Abilities of Large Language Modelscited by 4
  35. 2026Identifying and Transferring Reasoning-Critical Neurons: Improving LLM Inference Reliability via Activation Steeringcited by 4
  36. 2026DiscoLoop: Looping Discrete Embeddings and Continuous Hidden States for Multi-hop Reasoningcited by 4
  37. 2026Loop the Loopies!cited by 4
  38. 2025Are Transformers Able to Reason by Connecting Separated Knowledge in Training Data?cited by 3

50 most cited of 78 All 78 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Retrieval-enhanced Knowledge Editing in Language Models for Multi-Hop Question Answering
  2. 2025Unveiling the Mechanisms of Multi-Hop Reasoning in Transformers via Identity Bridge
  3. 2025KG-o1: Enhancing Multi-hop Question Answering in Large Language Models via Knowledge Graph Integration
  4. 2024Seemingly Plausible Distractors in Multi-Hop Reasoning: Are Large Language Models Attentive Readers?
  5. 2023Multimodal Multi-Hop Question Answering Through a Conversation Between Tools and Efficiently Finetuned Large Language Models
  6. 2023Leveraging Structured Information for Explainable Multi-hop Question Answering and Reasoning
  7. 2025Auto-Patching: Enhancing Multi-Hop Reasoning in Language Models
  8. 2024MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries