mlsanityData to 14 Oct 2026

Bucketing the Good Apples: A Method for Diagnosing and Improving Causal Abstraction

arXiv 2605.02234 ↗First posted 4 May 2026cs.AI cs.CLcited by 1

Not on the map. Why?

Abstract (arXiv)

We present a method for diagnosing interpretation in neural networks by identifying an input subspace where a proposed interpretation is highly faithful. Our method is particularly useful for causal-abstraction-style interpretability, where a high-level causal hypothesis is evaluated by interchange interventions. Rather than treating interchange intervention accuracy as a single global summary, we refine this framework by partitioning the input space into well-interpreted and under-interpreted regions according to pairwise interchange-intervention behavior. This turns causal abstraction from a purely global evaluation into a more diagnostic tool: it not only measures whether an interpretation works, but also reveals where it works, where it fails, and what distinguishes the two cases. This diagnostic view also provides practical heuristics for improving interpretations. By analyzing the structure of the well-interpreted and under-interpreted regions, we can identify missing distinctions in a high-level hypothesis, discover previously unmodeled intermediate variables, and combine complementary partial interpretations into a stronger one. We instantiate this idea as a simple four-step recipe and show that it yields informative error analyses across multiple causal abstraction settings. In a toy logic task, recursively applying the recipe recovers a high-level hypothesis from scratch. More broadly, our results suggest that partitioning the input space is a useful step toward more precise, constructive, and scalable mechanistic interpretability.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 25 of its 31 references that are in the map. First posted on arXiv 4 May 2026.

Builds on

25 of 31 references.

References · 25

  1. 2017Axiomatic Attribution for Deep Networkscited by 3,384
  2. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  3. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,420
  4. 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,336
  5. 2022Mass-Editing Memory in a Transformercited by 1,001
  6. 2022Toy Models of Superpositioncited by 971
  7. 2023Dissecting Recall of Factual Associations in Auto-Regressive Language Modelscited by 528
  8. 2021Causal Abstractions of Neural Networkscited by 422
  9. 2024Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2cited by 394
  10. 2023Finding Alignments Between Interpretable Causal Variables and Distributed Neural Representationscited by 209
  11. 2023Causal Abstraction: A Theoretical Foundation for Mechanistic Interpretabilitycited by 205
  12. 2023Linearity of Relation Decoding in Transformer Language Modelscited by 200
13 more
  1. 2024Llama Scope: Extracting Millions of Features from Llama-3.1-8B with Sparse Autoencoderscited by 143
  2. 2023Interpretability at Scale: Identifying Causal Mechanisms in Alpacacited by 103
  3. 2023Is This the Subspace You Are Looking for? An Interpretability Illusion for Subspace Activation Patchingcited by 80
  4. 2024RAVEL: Evaluating Interpretability Methods on Disentangling Language Model Representationscited by 79
  5. 2022Polysemanticity and Capacity in Neural Networkscited by 77
  6. 2014Visual Causal Feature Learningcited by 59
  7. 2024pyvene: A Library for Understanding and Improving PyTorch Models via Interventionscited by 54
  8. 2024CausalGym: Benchmarking causal interpretability methods on linguistic taskscited by 46
  9. 2025Everything, Everywhere, All at Once: Is Mechanistic Interpretability Identifiable?cited by 34
  10. 2025Mixing Mechanisms: How Language Models Retrieve Bound Entities In-Contextcited by 26
  11. 2025Causal Interventions Reveal Shared Structure Across English Filler-Gap Constructionscited by 21
  12. 2025Internal Causal Mechanisms Robustly Predict Language Model Out-of-Distribution Behaviorscited by 11
  13. 2025Combining Causal Models for More Accurate Abstractions of Neural Networkscited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Does the Model Use the Feature? Separating Steering from Mechanism in LLMscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025The Non-Linear Representation Dilemma: Is Causal Abstraction Enough for Mechanistic Interpretability?
  2. 2025Addressing divergent representations from causal interventions on neural networks
  3. 2024Causal Interventions on Causal Paths: Mapping GPT-2's Reasoning From Syntax to Semantics
  4. 2024The Quest for the Right Mediator: Surveying Mechanistic Interpretability Through the Lens of Causal Mediation Analysis
  5. 2026The Abstraction Gap in Vision-Language Causal Reasoning
  6. 2024Causal Concept Graph Models: Beyond Causal Opacity in Deep Learning
  7. 2025Causality can systematically address the monsters under the bench(marks)
  8. 2026Causality is Key for Interpretability Claims to Generalise