mlsanityData to 7 Oct 2026

Understanding and Patching Compositional Reasoning in LLMs

arXiv 2402.14328 ↗First posted 22 Feb 2024cs.CLcited by 45

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

LLMs have marked a revolutonary shift, yet they falter when faced with compositional reasoning tasks. Our research embarks on a quest to uncover the root causes of compositional reasoning failures of LLMs, uncovering that most of them stem from the improperly generated or leveraged implicit reasoning results. Inspired by our empirical findings, we resort to Logit Lens and an intervention experiment to dissect the inner hidden states of LLMs. This deep dive reveals that implicit reasoning results indeed surface within middle layers and play a causative role in shaping the final explicit reasoning results. Our exploration further locates multi-head self-attention (MHSA) modules within these layers, which emerge as the linchpins in accurate generation and leveraing of implicit reasoning results. Grounded on the above findings, we develop CREME, a lightweight method to patch errors in compositional reasoning via editing the located MHSA modules. Our empirical evidence stands testament to CREME's effectiveness, paving the way for autonomously and continuously enhancing compositional reasoning capabilities in language models.

Key facts. Cited by 45 papers in the map (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 10 of its 52 references that are in the map. First posted on arXiv 22 Feb 2024.

Builds on

10 of 52 references.

Closest ancestors · 1

Same direction, newest first

  1. 2021Show Your Work: Scratchpads for Intermediate Computation with Language Models30 Nov

Other references · 9

  1. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  2. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  3. 2020GLU Variants Improve Transformercited by 1,724
  4. 2023Eliciting Latent Predictions from Transformers with the Tuned Lenscited by 579
  5. 2024Patchscopes: A Unifying Framework for Inspecting Hidden Representations of Language Modelscited by 225
  6. 2024A Comprehensive Study of Knowledge Editing for Large Language Modelscited by 157
  7. 2022Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networkscited by 126
  8. 2023Identifying Linear Relational Concepts in Large Language Modelscited by 12
  9. 2023KLoB: a Benchmark for Assessing Knowledge Locating Methods in Language Modelscited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 13; 2025: 20; 2026: 12.

45 papers.

Sort:

By area:

  1. 2024Do Large Language Models Latently Perform Multi-Hop Reasoning?cited by 165
  2. 2024Hopping Too Late: Exploring the Limitations of Large Language Models on Multi-Hop Queriescited by 78
  3. 2025A Survey on Latent Reasoningcited by 71
  4. 2024Disentangling Memory and Reasoning Ability in Large Language Modelscited by 43
  5. 2024Decoding by Contrasting Knowledge: Enhancing LLMs' Confidence on Edited Factscited by 36
  6. 2024Locate-then-edit for Multi-hop Factual Recall under Knowledge Editingcited by 36
  7. 2024Reasoning Abilities of Large Language Models: In-Depth Analysis on the Abstraction and Reasoning Corpuscited by 32
  8. 2025Detection and Mitigation of Hallucination in Large Reasoning Models: A Mechanistic Perspectivecited by 32
  9. 2024Causal Prompting: Debiasing Large Language Model Prompting based on Front-Door Adjustmentcited by 29
  10. 2024Do Large Language Models Perform Latent Multi-Hop Reasoning without Exploiting Shortcuts?cited by 26
  11. 2025Back Attention: Understanding and Enhancing Multi-Hop Reasoning in Large Language Modelscited by 23
  12. 2025LogitLens4LLMs: Extending Logit Lens Analysis to Modern Large Language Modelscited by 22
33 more
  1. 2026Large Language Model Reasoning Failurescited by 20
  2. 2024Mitigating the Language Mismatch and Repetition Issues in LLM-based Machine Translation via Model Editingcited by 18
  3. 2025Unlocking the Power of Function Vectors for Characterizing and Mitigating Catastrophic Forgetting in Continual Instruction Tuningcited by 16
  4. 2024Understanding the Language Model to Solve the Symbolic Multi-Step Reasoning Problem from the Perspective of Buffer Mechanismcited by 15
  5. 2024Unveiling Factual Recall Behaviors of Large Language Models through Knowledge Neuronscited by 15
  6. 2024Understanding the Interplay between Parametric and Contextual Knowledge for Large Language Modelscited by 14
  7. 2025Credible Plan-Driven RAG Method for Multi-Hop Question Answeringcited by 12
  8. 2025When LLM Meets Time Series: Can LLMs Perform Multi-Step Time Series Reasoning and Inferencecited by 12
  9. 2025Compositional Generalization from Learned Skills via CoT Training: A Theoretical and Structural Analysis for Reasoningcited by 10
  10. 2026Do Latent-CoT Models Think Step-by-Step? A Mechanistic Study on Sequential Reasoning Taskscited by 10
  11. 2025CaKE: Circuit-aware Editing Enables Generalizable Knowledge Learnerscited by 9
  12. 2025How Do Language Models Compose Functions?cited by 7
  13. 2025Local Success Does Not Compose: Benchmarking Large Language Models for Compositional Formal Verificationcited by 6
  14. 2026Fine-Grained Activation Steering: Steering Less, Achieving Morecited by 6
  15. 2025CryptoX : Compositional Reasoning Evaluation of Large Language Modelscited by 5
  16. 2025The Thin Line Between Comprehension and Persuasion in LLMscited by 5
  17. 2025All for One: LLMs Solve Mental Math at the Last Token With Information Transferred From Other Tokenscited by 5
  18. 2025Understanding Subword Compositionality of Large Language Modelscited by 3
  19. 2025AgentCoMa: A Compositional Benchmark Mixing Commonsense and Mathematical Reasoning in Real-World Scenarioscited by 3
  20. 2025Agent-ScanKit: Unraveling Memory and Reasoning of Multimodal Agents via Sensitivity Perturbationscited by 3
  21. 2024From Words to Worlds: Compositionality for Cognitive Architecturescited by 1
  22. 2025On the Effect of Uncertainty on Layer-wise Inference Dynamicscited by 1
  23. 2026Layer-Order Inversion: Rethinking Latent Multi-Hop Reasoning in Large Language Modelscited by 1
  24. 2025Concept Attractors in LLMs and their Applicationscited by 1
  25. 2025Limit Analysis for Symbolic Multi-step Reasoning Tasks with Information Propagation Rules Based on Transformerscited by 0
  26. 2026Tracing Pharmacological Knowledge In Large Language Modelscited by 0
  27. 2026Evaluating Adjective-Noun Compositionality in LLMs: Functional vs Representational Perspectivescited by 0
  28. 2026How Psychological Learning Paradigms Shaped and Constrained Artificial Intelligencecited by 0
  29. 2026The Power of Power Law: Asymmetry Enables Compositional Reasoningcited by 0
  30. 2026Investigating More Explainable and Partition-Free Compositionality Estimation for LLMs: A Rule-Generation Perspectivecited by 0
  31. 2026Framing Matters: Addressing Framing Sensitivity in Decision-Making through Behaviorally-Grounded Value Alignmentcited by 0
  32. 2026Multi-Hop Knowledge Composition is Bound by Pretraining Exposurecited by 0
  33. 2026Why Knowing Both Hops Is Not Enough: Understanding Two-Hop Generalization in Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Reasoning or a Semblance of it? A Diagnostic Study of Transitive Reasoning in LLMs
  2. 2024Benchmarking and Understanding Compositional Relational Reasoning of LLMs
  3. 2024Do Large Language Models Have Compositional Ability? An Investigation into Limitations and Scalability
  4. 2025Dissecting Logical Reasoning in LLMs: A Fine-Grained Evaluation and Supervision Study
  5. 2025Implicit Reasoning in Large Language Models: A Comprehensive Survey
  6. 2025Comprehension Without Competence: Architectural Limits of LLMs in Symbolic Computation and Reasoning
  7. 2026Beyond Memorization: Testing LLM Reasoning on Unseen Theory of Computation Tasks
  8. 2023Skills-in-Context Prompting: Unlocking Compositionality in Large Language Models