iSAGE: An Incremental Version of SAGE for Online Explanation on Data Streams
Not on the map. Why?
Abstract (arXiv)
Existing methods for explainable artificial intelligence (XAI), including popular feature importance measures such as SAGE, are mostly restricted to the batch learning scenario. However, machine learning is often applied in dynamic environments, where data arrives continuously and learning must be done in an online manner. Therefore, we propose iSAGE, a time- and memory-efficient incrementalization of SAGE, which is able to react to changes in the model as well as to drift in the data-generating process. We further provide efficient feature removal methods that break (interventional) and retain (observational) feature dependencies. Moreover, we formally analyze our explanation method to show that iSAGE adheres to similar theoretical properties as SAGE. Finally, we evaluate our approach in a thorough experimental analysis based on well-established data sets and data streams with concept drift.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 22 of its 60 references that are in the map. First posted on arXiv 2 Mar 2023.
Builds on
22 of 60 references.
References · 22
- 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
- 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
- 2017Axiomatic Attribution for Deep Networkscited by 3,350
- 2013Visualizing and Understanding Convolutional Networkscited by 2,278
- 2017Learning Important Features Through Propagating Activation Differencescited by 1,454
- 2014Striving for Simplicity: The All Convolutional Netcited by 983
- 2020Explaining by Removing: A Unified Framework for Model Explanationcited by 218
- 2019Explaining individual predictions when features are dependent: More accurate approximations to Shapley valuescited by 205
- 2016Layer-wise Relevance Propagation for Neural Networks with Local Renormalization Layerscited by 197
- 2019Feature relevance quantification in explainable AI: A causal problemcited by 164
- 2020True to the Model or True to the Data?cited by 123
- 2020River: machine learning for streaming data in Pythoncited by 96
10 more
- 2020Shapley explainability on the data manifoldcited by 88
- 2019OpenML-Python: an extensible Python API for OpenMLcited by 62
- 2018Visualizing the Feature Importance for Black Box Modelscited by 55
- 2020Model-agnostic Feature Importance and Effects with Dependent Features -- A Conditional Subgroup Approachcited by 37
- 2020Sequential Deep Learning for Credit Risk Monitoring with Tabular Financial Datacited by 37
- 2020Relative Feature Importancecited by 12
- 2022Incremental Permutation Feature Importance (iPFI): Towards Online Explanations on Data Streamscited by 8
- 2022Change Detection for Local Explainability in Evolving Data Streamscited by 7
- 2018Sensitivity based Neural Networks Explanationscited by 5
- 2018Online Feature Ranking for Intrusion Detection Systemscited by 1
Built on by
Citing papers per year: 2023: 1; 2024: 1.
2 papers.