mlsanityData to 13 Oct 2026

Knowledge Vector of Logical Reasoning in Large Language Models

arXiv 2604.23877 ↗First posted 26 Apr 2026cs.CLcited by 0

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

Logical reasoning serve as a central capability in LLMs and includes three main forms: deductive, inductive, and abductive reasoning. In this work, we study the knowledge representations of these reasoning types in LLMs and analyze the correlations among them. Our analysis shows that each form of logical reasoning can be captured as a reasoning-specific knowledge vector in a linear representation space, yet these vectors are largely independent of each other. Motivated by cognitive science theory that these subforms of logical reasoning interact closely in the human brain, as well as our observation that the reasoning process for one type can benefit from the reasoning chain produced by another, we further propose to refine the knowledge representations of each reasoning type in LLMs to encourage complementarity between them. To this end, we design a complementary subspace-constrained refinement framework, which introduces a complementary loss that enables each reasoning vector to leverage auxiliary knowledge from the others, and a subspace constraint loss that prevents erasure of their unique characteristics. Through steering experiments along reasoning vectors, we find that refined vectors incorporating complementary knowledge yield consistent performance gains. We also conduct a mechanism-interpretability analysis of each reasoning vector, revealing insights into the shared and specific features of different reasoning in LLMs.

Key facts. No citing papers in the map yet (data through 13 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 31 of its 60 references that are in the map. First posted on arXiv 26 Apr 2026.

Builds on

31 of 60 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025Theorem-of-Thought: A Multi-Agent Framework for Abductive, Deductive, and Inductive Reasoning in Language Models8 Jun
  2. 2025Patterns Over Principles: The Fragility of Inductive Reasoning in LLMs under Noisy Observations22 Feb
  3. 2025JustLogic: A Comprehensive Benchmark for Evaluating Deductive Reasoning in Large Language Models24 Jan
  4. 2024TypedThinker: Diversify Large Language Model Reasoning with Typed Thinking2 Oct
  5. 2024Inductive or Deductive? Rethinking the Fundamental Reasoning Abilities of LLMs31 Jul

Other references · 26

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,093
  2. 2023GPT-4 Technical Reportcited by 18,278
  3. 2024The Llama 3 Herd of Modelscited by 14,486
  4. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,968
  5. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,391
  6. 2025gpt-oss-120b & gpt-oss-20b Model Cardcited by 1,319
  7. 2016Understanding intermediate layers using linear classifier probescited by 1,313
  8. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,286
  9. 2022Mass-Editing Memory in a Transformercited by 990
  10. 2022Toy Models of Superpositioncited by 950
  11. 2023Steering Language Models With Activation Engineeringcited by 873
  12. 2023The Linear Representation Hypothesis and the Geometry of Large Language Modelscited by 706
14 more
  1. 2024Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2cited by 386
  2. 2023Towards Best Practices of Activation Patching in Language Models: Metrics and Methodscited by 345
  3. 2019Abductive Commonsense Reasoningcited by 300
  4. 2024How to use and interpret activation patchingcited by 202
  5. 2024Improving Instruction-Following in Language Models through Activation Steeringcited by 153
  6. 2024Llama Scope: Extracting Millions of Features from Llama-3.1-8B with Sparse Autoencoderscited by 137
  7. 2025A Survey on Sparse Autoencoders: Interpreting the Internal Mechanisms of Large Language Modelscited by 75
  8. 2025Understanding Reasoning in Thinking Language Models via Steering Vectorscited by 71
  9. 2023Improving Activation Steering in Language Models with Mean-Centringcited by 65
  10. 2025Multi-Attribute Steering of Language Models via Targeted Interventioncited by 45
  11. 2024Multi-property Steering of Large Language Models with Dynamic Activation Compositioncited by 38
  12. 2025Model Unlearning via Sparse Autoencoder Subspace Guided Projectionscited by 17
  13. 2025Executable Counterfactuals: Improving LLMs' Causal Reasoning Through Codecited by 10
  14. 2025Steering LVLMs via Sparse Autoencoder for Hallucination Mitigationcited by 6

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Discovering a Shared Logical Subspace: Steering LLM Logical Reasoning via Alignment of Natural-Language and Symbolic Views
  2. 2023Towards LogiGLUE: A Brief Survey and A Benchmark for Analyzing Logical Reasoning Capabilities of Language Models
  3. 2023DetermLR: Augmenting LLM-based Logical Reasoning from Indeterminacy to Determinacy
  4. 2023Logical Reasoning over Natural Language as Knowledge Representation: A Survey
  5. 2025Logical Reasoning in Large Language Models: A Survey
  6. 2025Improving Reasoning Performance in Large Language Models via Representation Engineering
  7. 2024Disentangling Logic: The Role of Context in Large Language Model Reasoning Capabilities
  8. 2025Dissecting Logical Reasoning in LLMs: A Fine-Grained Evaluation and Supervision Study