mlsanityData to 14 Oct 2026

OrdShap: Feature Position Importance for Sequential Black-Box Models

arXiv 2507.11855 ↗First posted 16 Jul 2025cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Sequential deep learning models excel in domains with temporal or sequential dependencies, but their complexity necessitates post-hoc feature attribution methods for understanding their predictions. While existing techniques quantify feature importance, they inherently assume fixed feature ordering - conflating the effects of (1) feature values and (2) their positions within input sequences. To address this gap, we introduce OrdShap, a novel attribution method that disentangles these effects by quantifying how a model's predictions change in response to permuting feature position. We establish a game-theoretic connection between OrdShap and Sanchez-Berganti~nos values, providing a theoretically grounded approach to position-sensitive attribution. Empirical results from health, natural language, and synthetic datasets highlight OrdShap's effectiveness in capturing feature value and feature position attributions, and provide deeper insight into model behavior.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 58 of its 90 references that are in the map. First posted on arXiv 16 Jul 2025.

Builds on

58 of 90 references.

References · 58

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2017A Unified Approach to Interpreting Model Predictionscited by 6,555
  4. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,070
  5. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,307
  6. 2017Axiomatic Attribution for Deep Networkscited by 3,384
  7. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,218
  8. 2019Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AIcited by 1,552
  9. 2017Learning Important Features Through Propagating Activation Differencescited by 1,458
  10. 2015Show, Attend and Tell: Neural Image Caption Generation with Visual Attentioncited by 1,423
  11. 2017SmoothGrad: removing noise by adding noisecited by 1,109
  12. 2018A Survey Of Methods For Explaining Black Box Modelscited by 973
38 more
  1. 2018RISE: Randomized Input Sampling for Explanation of Black-box Modelscited by 838
  2. 2020Quantifying Attention Flow in Transformerscited by 775
  3. 2016Distribution-Free Predictive Inference For Regressioncited by 668
  4. 2020Transformer Interpretability Beyond Attention Visualizationcited by 627
  5. 2020Captum: A unified and generic model interpretability library for PyTorchcited by 547
  6. 2022Transformers in Time Series: A Surveycited by 497
  7. 2019Fooling LIME and SHAP: Adversarial Attacks on Post hoc Explanation Methodscited by 443
  8. 2020Med-BERT: pre-trained contextualized embeddings on large-scale structured electronic health records for disease predictioncited by 373
  9. 2019The many Shapley values for model explanationcited by 344
  10. 2018On the Robustness of Interpretability Methodscited by 314
  11. 2020A Survey on Neural Network Interpretabilitycited by 291
  12. 2018Consistent Individualized Feature Attribution for Tree Ensemblescited by 285
  13. 2019BEHRT: Transformer for Electronic Health Recordscited by 263
  14. 2020Explaining by Removing: A Unified Framework for Model Explanationcited by 218
  15. 2020Problems with Shapley-value-based explanations as feature importance measurescited by 200
  16. 2020Evaluating and Aggregating Feature-based Model Explanationscited by 188
  17. 2020Improving KernelSHAP: Practical Shapley Value Estimation via Linear Regressioncited by 154
  18. 2019The Shapley Taylor Interaction Indexcited by 143
  19. 2020Reliable Post hoc Explanations: Modeling Uncertainty in Explainabilitycited by 125
  20. 2021FastSHAP: Real-Time Shapley Value Estimationcited by 125
  21. 2019Explaining Deep Neural Networks with a Polynomial Time Algorithm for Shapley Values Approximationcited by 115
  22. 2022Algorithms to estimate Shapley value feature attributionscited by 115
  23. 2019Asymmetric Shapley values: incorporating causal knowledge into model-agnostic explainabilitycited by 114
  24. 2020Explaining Explanations: Axiomatic Feature Interactions for Deep Networkscited by 103
  25. 2021Sampling Permutations for Shapley Value Estimationcited by 102
  26. 2022The Shapley Value in Machine Learningcited by 98
  27. 2021CEHR-BERT: Incorporating temporal information from structured EHR data to improve prediction taskscited by 76
  28. 2020TimeSHAP: Explaining Recurrent Models through Sequence Perturbationscited by 74
  29. 2020On quantitative aspects of model interpretabilitycited by 71
  30. 2020Efficient nonparametric statistical inference on population feature importance using Shapley valuescited by 61
  31. 2020Shapley Flow: A Graph-based Approach to Interpreting Model Predictionscited by 51
  32. 2023SHAP-IQ: Unified Approximation of any-order Shapley Interactionscited by 51
  33. 2020Statistical stability indices for LIME: obtaining reliable explanations for Machine Learning modelscited by 40
  34. 2021Attention Flows are Shapley Value Explanationscited by 30
  35. 2020Transparency, Auditability and eXplainability of Machine Learning Models in Credit Scoringcited by 26
  36. 2021Unifying Heterogeneous Electronic Health Records Systems via Text-Based Code Embeddingcited by 24
  37. 2024KernelSHAP-IQ: Weighted Least-Square Optimization for Shapley Interactionscited by 22
  38. 2024SVARM-IQ: Efficient Approximation of Any-order Shapley Interactions through Stratificationcited by 16

50 most cited of 58 All 58 (JSON)

Built on by

Not cited yet.