mlsanityData to 14 Oct 2026

DePass: Unified Feature Attributing by Simple Decomposed Forward Pass

arXiv 2510.18462 ↗First posted 21 Oct 2025cs.CLcited by 1

On the map: Mechanistic interpretability → Circuits and mechanistic analysis

Abstract (arXiv)

Attributing the behavior of Transformer models to internal computations is a central challenge in mechanistic interpretability. We introduce DePass, a unified framework for feature attribution based on a single decomposed forward pass. DePass decomposes hidden states into customized additive components, then propagates them with attention scores and MLP's activations fixed. It achieves faithful, fine-grained attribution without requiring auxiliary training. We validate DePass across token-level, model component-level, and subspace-level attribution tasks, demonstrating its effectiveness and fidelity. Our experiments highlight its potential to attribute information flow between arbitrary components of a Transformer model. We hope DePass serves as a foundational tool for broader applications in interpretability.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Circuits and mechanistic analysis, one of 13 directions in Mechanistic interpretability. Builds on 46 of its 52 references that are in the map. First posted on arXiv 21 Oct 2025.

Builds on

46 of 52 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025Open Problems in Mechanistic Interpretability27 Jan
  2. 2024Transcoders Find Interpretable LLM Feature Circuits17 Jun
  3. 2023Is This the Subspace You Are Looking for? An Interpretability Illusion for Subspace Activation Patching28 Nov
  4. 2023Attribution Patching Outperforms Automated Circuit Discovery16 Oct
  5. 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small1 Nov

Other references · 41

  1. 2017A Unified Approach to Interpreting Model Predictionscited by 6,555
  2. 2017Axiomatic Attribution for Deep Networkscited by 3,384
  3. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,975
  4. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  5. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,420
  6. 2016Understanding intermediate layers using linear classifier probescited by 1,353
  7. 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,336
  8. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,302
  9. 2022Toy Models of Superpositioncited by 971
  10. 2020Quantifying Attention Flow in Transformerscited by 775
  11. 2024Scaling and evaluating sparse autoencoderscited by 553
  12. 2022Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Spacecited by 532
29 more
  1. 2018What you can cram into a single vector: Probing sentence embeddings for linguistic propertiescited by 457
  2. 2023Finding Neurons in a Haystack: Case Studies with Sparse Probingcited by 334
  3. 2024Llama Scope: Extracting Millions of Features from Llama-3.1-8B with Sparse Autoencoderscited by 143
  4. 2024AttnLRP: Attention-Aware Layer-Wise Relevance Propagation for Transformerscited by 120
  5. 2020Learning to Faithfully Rationalize by Constructioncited by 113
  6. 2024AtP*: An efficient and scalable method for localizing LLM behaviour to componentscited by 111
  7. 2022XAI for Transformers: Better Explanations through Conservative Propagationcited by 100
  8. 2024ContextCite: Attributing Model Generation to Contextcited by 98
  9. 2024Information Flow Routes: Automatically Interpreting Language Models at Scalecited by 95
  10. 2024Improving Steering Vectors by Targeting Sparse Autoencoder Featurescited by 92
  11. 2024A Primer on the Inner Workings of Transformer-based Language Modelscited by 85
  12. 2022Impossibility Theorems for Feature Attributioncited by 78
  13. 2024SelfIE: Self-Interpretation of Large Language Model Embeddingscited by 67
  14. 2022Measuring the Mixing of Contextual Information in the Transformercited by 65
  15. 2014Extraction of Salient Sentences from Labelled Documentscited by 50
  16. 2024Decomposing The Dark Matter of Sparse Autoencoderscited by 45
  17. 2023Quantifying Context Mixing in Transformerscited by 43
  18. 2023Explaining How Transformers Use Context to Build Predictionscited by 43
  19. 2023Analyzing Feed-Forward Blocks in Transformers through the Lens of Attention Mapscited by 33
  20. 2023DecompX: Explaining Transformers Decisions by Propagating Token Decompositioncited by 31
  21. 2024Knowledge Overshadowing Causes Amalgamated Hallucination in Large Language Modelscited by 25
  22. 2024Mechanistic Permutability: Match Features Across Layerscited by 22
  23. 2025SelfCite: Self-Supervised Alignment for Context Attribution in Large Language Modelscited by 20
  24. 2024Truth-Aware Context Selection: Mitigating Hallucinations of Large Language Models Being Misled by Untruthful Contextscited by 15
  25. 2024Evolution of SAE Features Across Layers in LLMscited by 14
  26. 2025Analyze Feature Flow to Enhance Interpretation and Steering in Language Modelscited by 13
  27. 2025A Close Look at Decomposition-based XAI-Methods for Transformer Language Modelscited by 11
  28. 2023Token-wise Decomposition of Autoregressive Language Model Hidden States for Analyzing Model Predictionscited by 6
  29. 2025Noiser: Bounded Input Perturbations for Attributing Large Language Modelscited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Layer-wise Target Propagation: Efficient Component Attribution through Target Centric Propagationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Hidden Heroes and Gradient Bloats: Layer-Wise Redundancy Inverts Attribution in Transformers
  2. 2026Every Component Is a Lookup: One Linear Graph for Interaction, Composition and Attribution
  3. 2025From Concepts to Components: Concept-Agnostic Attention Module Discovery in Transformers
  4. 2026From Correlation to Cause: A Five-Stage Methodology for Feature Analysis in Transformer Language Models
  5. 2025Beyond Components: Singular Vector-Based Interpretability of Transformer Circuits
  6. 2025Interpreting vision transformers via residual replacement model
  7. 2026ResLRP: The Role of Residual Cancellation in Attribution Instability in Vision Transformers
  8. 2026Fully Interpretable Minimal Transformers: From Geometry to Algorithm