mlsanityData to 7 Oct 2026

Rethinking Circuit Completeness in Language Models: AND, OR, and ADDER Gates

arXiv 2505.10039 ↗First posted 15 May 2025cs.LGcited by 8

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

Circuit discovery has gradually become one of the prominent methods for mechanistic interpretability, and research on circuit completeness has also garnered increasing attention. Methods of circuit discovery that do not guarantee completeness not only result in circuits that are not fixed across different runs but also cause key mechanisms to be omitted. The nature of incompleteness arises from the presence of OR gates within the circuit, which are often only partially detected in standard circuit discovery methods. To this end, we systematically introduce three types of logic gates: AND, OR, and ADDER gates, and decompose the circuit into combinations of these logical gates. Through the concept of these gates, we derive the minimum requirements necessary to achieve faithfulness and completeness. Furthermore, we propose a framework that combines noising-based and denoising-based interventions, which can be easily integrated into existing circuit discovery methods without significantly increasing computational complexity. This framework is capable of fully identifying the logic gates and distinguishing them within the circuit. In addition to the extensive experimental validation of the framework's ability to restore the faithfulness, completeness, and sparsity of circuits, using this framework, we uncover fundamental properties of the three logic gates, such as their proportions and contributions to the output, and explore how they behave among the functionalities of language models.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 23 of its 26 references that are in the map. First posted on arXiv 15 May 2025.

Builds on

23 of 26 references.

References · 23

  1. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  2. 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,130
  3. 2023BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Datasetcited by 828
  4. 2023Towards Automated Circuit Discovery for Mechanistic Interpretabilitycited by 700
  5. 2024Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Modelscited by 412
  6. 2023Towards Best Practices of Activation Patching in Language Models: Metrics and Methodscited by 328
  7. 2023How does GPT-2 compute greater-than?: Interpreting mathematical abilities in a pre-trained language modelcited by 199
  8. 2023Attribution Patching Outperforms Automated Circuit Discoverycited by 196
  9. 2024How to use and interpret activation patchingcited by 191
  10. 2023Localizing Model Behavior with Path Patchingcited by 182
  11. 2023Does Circuit Analysis Interpretability Scale? Evidence from Multiple Choice Capabilities in Chinchillacited by 145
  12. 2021Comparing Kullback-Leibler Divergence and Mean Squared Error Loss in Knowledge Distillationcited by 131
11 more
  1. 2024The Evolution of Statistical Induction Heads: In-Context Learning Markov Chainscited by 115
  2. 2024A Practical Review of Mechanistic Interpretability for Transformer-Based Language Modelscited by 113
  3. 2024Knowledge Circuits in Pretrained Transformerscited by 60
  4. 2024Finding Transformer Circuits with Edge Pruningcited by 58
  5. 2024Identifying Semantic Induction Heads to Understand In-Context Learningcited by 46
  6. 2024Induction Heads as an Essential Mechanism for Pattern Matching in In-context Learningcited by 46
  7. 2021Sparse Interventions in Language Models with Differentiable Maskingcited by 34
  8. 2024Efficient Automated Circuit Discovery in Transformers using Contextual Decompositioncited by 23
  9. 2024Missed Causes and Ambiguous Effects: Counterfactuals Pose Challenges for Interpreting Neural Networkscited by 21
  10. 2023Discovering Knowledge-Critical Subnetworks in Pretrained Language Modelscited by 16
  11. 2025Position-aware Automatic Circuit Discoverycited by 14

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 6.

8 papers.

  1. 2025Addressing divergent representations from causal interventions on neural networkscited by 6
  2. 2026Conditional Co-Ablation: Recovering Self-Repair Backups in Transformer Circuitscited by 6
  3. 2025Discovering Transformer Circuits via a Hybrid Attribution and Pruning Frameworkcited by 4
  4. 2026Navigating by Old Maps: The Pitfalls of Static Mechanistic Localization in LLM Post-Trainingcited by 3
  5. 2026Beyond Transfer Accuracy: Mechanism-Guided Controlled Adaptation for Low-Resource Languagescited by 0
  6. 2026Can Graph Learning Learn Circuits?cited by 0
  7. 2026Beyond Static Interpretability: Anticipating Post-SFT Mechanisms from Pre-SFT Parameters for Better Tuningcited by 0
  8. 2026RECLAIM: Can Agents Reproduce the Claims of Machine Learning Papers?cited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Can Language Model Agents be Helpful Circuit Explainers in Mechanistic Interpretability?
  2. 2025Circuit Stability Characterizes Language Model Generalization
  3. 2026All Circuits Lead to Rome: Rethinking Functional Anisotropy in Circuit and Sheaf Discovery for LLMs
  4. 2025LogicIF: Towards Complex Logic Instruction Following
  5. 2026LogicGraph : Benchmarking Multi-Path Logical Reasoning via Neuro-Symbolic Generation and Verification
  6. 2024A Implies B: Circuit Analysis in LLMs for Propositional Logical Reasoning
  7. 2025Transformers are Inherently Succinct
  8. 2026LogicSkills: A Structured Benchmark for Formal Reasoning in Large Language Models