mlsanityData to 14 Oct 2026

The Logical Implication Steering Method for Conditional Interventions on Transformer Generation

arXiv 2502.03618 ↗First posted 5 Feb 2025cs.LGcited by 1

On the map: Mechanistic interpretability → Circuits and mechanistic analysis

Abstract (arXiv)

The field of mechanistic interpretability in pre-trained transformer models has demonstrated substantial evidence supporting the ''linear representation hypothesis'', which is the idea that high level concepts are encoded as vectors in the space of activations of a model. Studies also show that model generation behavior can be steered toward a given concept by adding the concept's vector to the corresponding activations. We show how to leverage these properties to build a form of logical implication into models, enabling transparent and interpretable adjustments that induce a chosen generation behavior in response to the presence of any given concept. Our method, Logical Implication Model Steering (LIMS), unlocks new hand engineered reasoning capabilities by integrating neuro-symbolic logic into pre-trained transformer models.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Circuits and mechanistic analysis, one of 13 directions in Mechanistic interpretability. Builds on 24 of its 41 references that are in the map. First posted on arXiv 5 Feb 2025.

Builds on

24 of 41 references.

References · 24

  1. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  2. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  3. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  4. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
  5. 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,554
  6. 2017Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)cited by 1,319
  7. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 1,032
  8. 2022Mass-Editing Memory in a Transformercited by 1,001
  9. 2023Steering Language Models With Activation Engineeringcited by 922
  10. 2022Discovering Latent Knowledge in Language Models Without Supervisioncited by 767
  11. 2023The Linear Representation Hypothesis and the Geometry of Large Language Modelscited by 716
  12. 2024GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Modelscited by 550
12 more
  1. 2023Emergent Linear Representations in World Models of Self-Supervised Sequence Modelscited by 319
  2. 2023Physics of Language Models: Part 3.1, Knowledge Storage and Extractioncited by 270
  3. 2024Chain-of-Thought Reasoning Without Promptingcited by 210
  4. 2023Physics of Language Models: Part 3.2, Knowledge Manipulationcited by 143
  5. 2024Physics of Language Models: Part 2.1, Grade-School Math and the Hidden Reasoning Processcited by 108
  6. 2024ChatQA: Surpassing GPT-4 on Conversational QA and RAGcited by 90
  7. 2024RAVEL: Evaluating Interpretability Methods on Disentangling Language Model Representationscited by 79
  8. 2024Adaptive Activation Steering: A Tuning-Free LLM Truthfulness Improvement Method for Diverse Hallucinations Categoriescited by 77
  9. 2024The Geometry of Concepts: Sparse Autoencoder Feature Structurecited by 52
  10. 2021Logic-Driven Context Extension and Data Augmentation for Logical Reasoning of Textcited by 46
  11. 2024Interpreting Attention Layer Outputs with Sparse Autoencoderscited by 43
  12. 2025Mass-Editing Memory with Attention in Transformers: A cross-lingual exploration of knowledgecited by 10

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026A Survey on the Linear Representation Hypothesiscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Mechanistic Unveiling of Transformer Circuits: Self-Influence as a Key to Model Reasoning
  2. 2023Assessing Logical Reasoning Capabilities of Encoder-Only Transformer Models
  3. 2024Reasoning in Transformers -- Mitigating Spurious Correlations and Reasoning Shortcuts
  4. 2026To See the Unseen: on the Generalization Ability of Transformers in Symbolic Reasoning
  5. 2025Hierarchical Reasoning Models: Perspectives and Misconceptions
  6. 2026Emergent Analogical Reasoning in Transformers
  7. 2025Beyond Components: Singular Vector-Based Interpretability of Transformer Circuits
  8. 2026Align, Unify, Suppress, Route: A Coherentist View of Transformer Computation