mlsanityData to 14 Oct 2026

Generalized Attention Flow: Feature Attribution for Transformer Models via Maximum Flow

arXiv 2502.15765 ↗First posted 14 Feb 2025cs.LG cs.AIcited by 4

Not on the map. Why?

Abstract (arXiv)

This paper introduces Generalized Attention Flow (GAF), a novel feature attribution method for Transformer-based models to address the limitations of current approaches. By extending Attention Flow and replacing attention weights with the generalized Information Tensor, GAF integrates attention weights, their gradients, the maximum flow problem, and the barrier method to enhance the performance of feature attributions. The proposed method exhibits key theoretical properties and mitigates the shortcomings of prior techniques that rely solely on simple aggregation of attention weights. Our comprehensive benchmarking on sequence classification tasks demonstrates that a specific variant of GAF consistently outperforms state-of-the-art feature attribution methods in most evaluation settings, providing a more reliable interpretation of Transformer model outputs.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 35 of its 56 references that are in the map. First posted on arXiv 14 Feb 2025.

Builds on

35 of 56 references.

References · 35

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2017A Unified Approach to Interpreting Model Predictionscited by 6,555
  4. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,070
  5. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,548
  6. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,307
  7. 2017Axiomatic Attribution for Deep Networkscited by 3,384
  8. 2021Convex optimizationcited by 2,515
  9. 2015Character-level Convolutional Networks for Text Classificationcited by 2,330
  10. 2019Attention is not Explanationcited by 929
  11. 2019Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Prunedcited by 856
  12. 2020Quantifying Attention Flow in Transformerscited by 775
23 more
  1. 2016CVXPY: A Python-Embedded Modeling Language for Convex Optimizationcited by 641
  2. 2020Transformer Interpretability Beyond Attention Visualizationcited by 627
  3. 2014Convex Optimization: Algorithms and Complexitycited by 575
  4. 2020Captum: A unified and generic model interpretability library for PyTorchcited by 547
  5. 2019The many Shapley values for model explanationcited by 344
  6. 2021Generic Attention-model Explainability for Interpreting Bi-Modal and Encoder-Decoder Transformerscited by 320
  7. 2017A Rewriting System for Convex Optimization Problemscited by 179
  8. 2021FastSHAP: Real-Time Shapley Value Estimationcited by 125
  9. 2024Information Flow Routes: Automatically Interpreting Language Models at Scalecited by 95
  10. 2020Generating Hierarchical Explanations on Text Classification via Feature Interaction Detectioncited by 57
  11. 2021Incorporating Residual and Normalization Layers into Analysis of Masked Language Modelscited by 51
  12. 2022GlobEnc: Quantifying Global Token Attribution by Incorporating the Whole Encoder Layer in Transformerscited by 44
  13. 2022Grad-SAM: Explaining Transformers via Gradient Self-Attention Mapscited by 42
  14. 2017An optimal transportation approach for assessing almost stochastic ordercited by 33
  15. 2023DecompX: Explaining Transformers Decisions by Propagating Token Decompositioncited by 31
  16. 2021Attention Flows are Shapley Value Explanationscited by 30
  17. 2022deep-significance - Easy and Meaningful Statistical Significance Testing in the Age of Neural Networkscited by 28
  18. 2018Computationally Efficient Measures of Internal Neuron Importancecited by 20
  19. 2023HarsanyiNet: Computing Accurate Shapley Values in a Single Forward Propagationcited by 17
  20. 2019Solving Linear Programs with Sqrt(rank) Linear System Solvescited by 15
  21. 2023Almost-Linear Time Algorithms for Incremental Graphs: Cycle Detection, SCCs, s-t Shortest Path, and Minimum-Cost Flowcited by 4
  22. 2021Faster Maxflow via Improved Dynamic Spectral Vertex Sparsifierscited by 3
  23. 2021Faster Sparse Minimum Cost Flow by Electrical Flow Localizationcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026Locate, Steer, and Improve: A Practical Survey of Actionable Mechanistic Interpretability in Large Language Modelscited by 27
  2. 2026Explainable AI: Context-Aware Layer-Wise Integrated Gradients for Explaining Transformer Modelscited by 1
  3. 2026Localizing Prompt Ambiguity in Large Language Models with Probe-Targeted Attributioncited by 0
  4. 2026RIFT: Token Influence Follows the Path of Least Resistancecited by 0