mlsanityData to 14 Oct 2026

Beyond Importance: Interchange-Sobol Sensitivity Reveals Task-Specific Content Channels in Transformer Components

arXiv 2606.20678 ↗First posted 12 Jun 2026stat.ML cs.LG stat.MEcited by 3

On the map: Mechanistic interpretability → Circuits and mechanistic analysis

Abstract (arXiv)

Mechanistic interpretability methods summarize a transformer component by a single importance score, conflating two distinct roles: a component may matter because it transports task-relevant content, or because the forward computation degrades when its contribution is removed. We introduce Interchange-Group Sobol Decomposition (IGSD), a paired-intervention framework that compares matched activation replacement with zero ablation on the same component, estimates two Sobol-style variance indices, and uses their signed difference to separate the two roles, with intervention validity monitored by a symmetric off-manifold diagnostic widehatST>1. In factual recall, IGSD identifies an early-layer content channel in both GPT-2 small and Qwen2.5-1.5B that standard importance methods underestimate. A controlled subject and relation donor design shows that the early channel transports relation-frame content while late attention transports subject-retrieval content, refining at head granularity to the known AttnL9H8 head. Late-layer clamping confirms that the early signal is expressed through downstream transformations rather than residual pass-through. These results show that replacement and deletion are not interchangeable controls and their divergence provides a practical statistical diagnostic for content transport in transformer components.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Circuits and mechanistic analysis, one of 13 directions in Mechanistic interpretability. Builds on 21 of its 37 references that are in the map. First posted on arXiv 12 Jun 2026.

Builds on

21 of 37 references.

Closest ancestors · 6

Same direction, newest first

  1. 2025Circuit Insights: Towards Interpretability Beyond Activations16 Oct
  2. 2025The Non-Linear Representation Dilemma: Is Causal Abstraction Enough for Mechanistic Interpretability?11 Jul
  3. 2024Mechanistic Interpretability for AI Safety -- A Review22 Apr
  4. 2023Towards Automated Circuit Discovery for Mechanistic Interpretability28 Apr
  5. 2023Causal Abstraction: A Theoretical Foundation for Mechanistic Interpretability11 Jan
  6. 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small1 Nov

Other references · 15

  1. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  2. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,336
  3. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,302
  4. 2024Qwen2.5-Coder Technical Reportcited by 1,203
  5. 2023SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shotcited by 1,121
  6. 2022Mass-Editing Memory in a Transformercited by 1,001
  7. 2023Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generationcited by 819
  8. 2023A Simple and Effective Pruning Approach for Large Language Modelscited by 800
  9. 2023Dissecting Recall of Factual Associations in Auto-Regressive Language Modelscited by 528
  10. 2023Towards Best Practices of Activation Patching in Language Models: Metrics and Methodscited by 359
  11. 2024How to use and interpret activation patchingcited by 214
  12. 2023Linearity of Relation Decoding in Transformer Language Modelscited by 200
3 more
  1. 2026Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnetcited by 191
  2. 2024AtP*: An efficient and scalable method for localizing LLM behaviour to componentscited by 111
  3. 2024Extracting Interpretable Task-Specific Circuits from Large Language Models for Faster Inferencecited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026What Does Layer-Importance Reveal About Transformers and State-Space Models?
  2. 2026Every Component Is a Lookup: One Linear Graph for Interaction, Composition and Attribution
  3. 2025Causal Head Gating: A Framework for Interpreting Roles of Attention Heads in Transformers
  4. 2026Memory Retrieval in Transformers: Insights from The Encoding Specificity Principle
  5. 2025Disentangling Recall and Reasoning in Transformer Models through Layer-wise Attention and Activation Analysis
  6. 2026Contrastive Projection: Reading Transformer Internals by Differencing Logit Lenses
  7. 2025From Concepts to Components: Concept-Agnostic Attention Module Discovery in Transformers
  8. 2023Interpretability in Activation Space Analysis of Transformers: A Focused Survey