mlsanityData to 7 Oct 2026

Sheaf Discovery with Joint Computation Graph Pruning and Flexible Granularity

arXiv 2407.03779 ↗First posted 4 Jul 2024cs.CL cs.LGcited by 11

Not on the map. Why?

Abstract (arXiv)

In this paper, we introduce DiscoGP, a novel framework for extracting self-contained modular units, or sheaves, within neural language models (LMs). Sheaves extend the concept of functional circuits, a unit widely explored in interpretability research, by considering not only subsets of edges in an LM's computation graph but also the model's weight parameters. Our framework identifies sheaves through a gradient-based pruning algorithm that operates on both of these in such a way that reduces the original LM to a sparse skeleton that preserves certain core capabilities. Experimental results demonstrate that, across a range of linguistic and reasoning tasks, DiscoGP extracts sheaves that preserve 93%-100% of a model's performance on the identified task while comprising only 1%-7% of the original weights and connections. Furthermore, our analysis reveals that, compared to previously identified LM circuits, the sheaves discovered by DiscoGP exhibit superior modularity and functional fidelity. Extending our method to the neuron level also unveils novel insights into the inner workings of LLMs

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 33 of its 57 references that are in the map. First posted on arXiv 4 Jul 2024.

Builds on

33 of 57 references.

References · 33

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  5. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  6. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,250
  7. 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,031
  8. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,359
  9. 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,307
  10. 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,130
  11. 2023Towards Automated Circuit Discovery for Mechanistic Interpretabilitycited by 700
  12. 2017Learning Sparse Neural Networks through L0 Regularizationcited by 571
21 more
  1. 2019BLiMP: The Benchmark of Linguistic Minimal Pairs for Englishcited by 553
  2. 2022Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Spacecited by 522
  3. 2023Dissecting Recall of Factual Associations in Auto-Regressive Language Modelscited by 512
  4. 2024Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Modelscited by 412
  5. 2021Causal Abstractions of Neural Networkscited by 401
  6. 2021Measuring and Improving Consistency in Pretrained Language Modelscited by 361
  7. 2020Parameter-Efficient Transfer Learning with Diff Pruningcited by 351
  8. 2023Towards Best Practices of Activation Patching in Language Models: Metrics and Methodscited by 328
  9. 2023Linearity of Relation Decoding in Transformer Language Modelscited by 190
  10. 2023Localizing Model Behavior with Path Patchingcited by 182
  11. 2024Have Faith in Faithfulness: Going Beyond Circuit Overlap When Finding Model Mechanismscited by 144
  12. 2021Causal Analysis of Syntactic Agreement Mechanisms in Neural Language Modelscited by 127
  13. 2019Winning the Lottery with Continuous Sparsificationcited by 99
  14. 2023Tracr: Compiled Transformers as a Laboratory for Interpretabilitycited by 99
  15. 2020Are Neural Nets Modular? Inspecting Functional Modularity Through Differentiable Weight Maskscited by 97
  16. 2021Can Subnetwork Structure be the Key to Out-of-Distribution Generalization?cited by 75
  17. 2021Low-Complexity Probing via Finding Subnetworkscited by 63
  18. 2024Finding Transformer Circuits with Edge Pruningcited by 58
  19. 2024What does the Knowledge Neuron Thesis Have to do with Knowledge?cited by 49
  20. 2024Mechanistic Understanding and Mitigation of Language Model Non-Factual Hallucinationscited by 38
  21. 2021Sparse Interventions in Language Models with Differentiable Maskingcited by 34

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 6; 2026: 4.

11 papers.

  1. 2024Knowledge Mechanisms in Large Language Models: A Survey and Perspectivecited by 50
  2. 2025Mechanistic Interpretability as Statistical Estimation: A Variance Analysiscited by 14
  3. 2025UORA: Uniform Orthogonal Reinitialization Adaptation in Parameter-Efficient Fine-Tuning of Large Modelscited by 11
  4. 2026All Circuits Lead to Rome: Rethinking Functional Anisotropy in Circuit and Sheaf Discovery for LLMscited by 8
  5. 2025CLUE: Conflict-guided Localization for LLM Unlearning Frameworkcited by 7
  6. 2025Llama See, Llama Do: A Mechanistic Perspective on Contextual Entrainment and Distraction in LLMscited by 6
  7. 2025Granular Concept Circuits: Toward a Fine-Grained Circuit Discovery for Concept Representationscited by 5
  8. 2026Circuit Claims Depend on What Is Extracted and How It Is Comparedcited by 2
  9. 2025Multi-Granular Node Pruning for Causal Circuit Discoverycited by 0
  10. 2026Mechanistically Interpreting Compression in Vision-Language Modelscited by 0
  11. 2026Circuit Condensation: Post-Training that Concentrates a Behavior's Causal Circuitcited by 0