mlsanityData to 7 Oct 2026

Beyond Explaining: Opportunities and Challenges of XAI-Based Model Improvement

arXiv 2203.08008 ↗First posted 15 Mar 2022cs.LGcited by 48

Not on the map. Why?

Abstract (arXiv)

Explainable Artificial Intelligence (XAI) is an emerging research field bringing transparency to highly complex and opaque machine learning (ML) models. Despite the development of a multitude of methods to explain the decisions of black-box classifiers in recent years, these tools are seldomly used beyond visualization purposes. Only recently, researchers have started to employ explanations in practice to actually improve models. This paper offers a comprehensive overview over techniques that apply XAI practically for improving various properties of ML models, and systematically categorizes these approaches, comparing their respective strengths and weaknesses. We provide a theoretical perspective on these methods, and show empirically through experiments on toy and realistic settings how explanations can help improve properties such as model generalization ability or reasoning, among others. We further discuss potential caveats and drawbacks of these methods. We conclude that while model improvement based on XAI can have significant beneficial effects even on complex and not easily quantifyable model properties, these methods need to be applied carefully, since their success can vary depending on a multitude of factors, such as the model and dataset used, or the employed explanation method.

Key facts. Cited by 48 papers in the map (data through 7 Oct 2026). Builds on 73 of its 134 references that are in the map. First posted on arXiv 15 Mar 2022.

Builds on

73 of 134 references.

References · 73

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  4. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  5. 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
  6. 2013Intriguing properties of neural networkscited by 5,114
  7. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
  8. 2017Axiomatic Attribution for Deep Networkscited by 3,350
  9. 2015Learning Deep Features for Discriminative Localizationcited by 2,430
  10. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,348
  11. 2016Matching Networks for One Shot Learningcited by 2,246
  12. 2019Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AIcited by 1,543
38 more
  1. 2017Learning Important Features Through Propagating Activation Differencescited by 1,454
  2. 2017Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)cited by 1,309
  3. 2017A Survey on Deep Learning in Medical Image Analysiscited by 1,182
  4. 2017Learning to Compare: Relation Network for Few-Shot Learningcited by 1,139
  5. 2016Pruning Filters for Efficient ConvNetscited by 1,129
  6. 2014Striving for Simplicity: The All Convolutional Netcited by 983
  7. 2017Deep Bayesian Active Learning with Image Datacited by 812
  8. 2015The Limitations of Deep Learning in Adversarial Settingscited by 784
  9. 2018This Looks Like That: Deep Learning for Interpretable Image Recognitioncited by 731
  10. 2017Network Dissection: Quantifying Interpretability of Deep Visual Representationscited by 703
  11. 2019Skin Lesion Analysis Toward Melanoma Detection 2018: A Challenge Hosted by the International Skin Imaging Collaboration (ISIC)cited by 657
  12. 2017Interpretable Explanations of Black Boxes by Meaningful Perturbationcited by 611
  13. 2018Detecting Backdoor Attacks on Deep Neural Networks by Activation Clusteringcited by 429
  14. 2019A Survey on Explainable Artificial Intelligence (XAI): Towards Medical XAIcited by 403
  15. 2019Unmasking Clever Hans Predictors and Assessing What Machines Really Learncited by 359
  16. 2018Learning and Generalization in Overparameterized Neural Networks, Going Beyond Two Layerscited by 340
  17. 2017Few-Shot Learning with Graph Neural Networkscited by 324
  18. 2016Top-down Neural Attention by Excitation Backpropcited by 318
  19. 2017Right for the Right Reasons: Training Differentiable Models by Constraining their Explanationscited by 293
  20. 2017Improving the Adversarial Robustness and Interpretability of Deep Neural Networks by Regularizing their Input Gradientscited by 236
  21. 2018Women also Snowboard: Overcoming Bias in Captioning Modelscited by 236
  22. 2020Concept Whitening for Interpretable Image Recognitioncited by 225
  23. 2019BCN20000: Dermoscopic Lesions in the Wildcited by 224
  24. 2016Understanding Neural Networks through Representation Erasurecited by 223
  25. 2019Explanations can be manipulated and geometry is to blamecited by 215
  26. 2017Visualizing Deep Neural Network Decisions: Prediction Difference Analysiscited by 166
  27. 2019Interpretations are useful: penalizing explanations to align neural networks with prior knowledgecited by 144
  28. 2020Making deep neural networks right for the right scientific reasons by interacting with their explanationscited by 141
  29. 2019Taking a HINT: Leveraging Explanations to Make Vision and Language Models More Groundedcited by 127
  30. 2018Attention Branch Network: Learning of Attention Mechanism for Visual Explanationcited by 108
  31. 2020Right for the Right Concept: Revising Neuro-Symbolic Concepts by Interacting with their Explanationscited by 94
  32. 2019Pruning by Explaining: A Novel Criterion for Deep Neural Network Pruningcited by 89
  33. 2020Explainable Artificial Intelligence: a Systematic Reviewcited by 79
  34. 2021Unbox the Black-box for the Medical Explainable AI via Multi-modal and Multi-centre Data Fusion: A Mini-Review, Two Showcases and Beyondcited by 75
  35. 2019Summit: Scaling Deep Learning Interpretability by Visualizing Activation and Attribution Summarizationscited by 72
  36. 2020Fairwashing Explanations with Off-Manifold Detergentcited by 71
  37. 2019Incorporating Priors with Feature Attribution on Text Classificationcited by 68
  38. 2021Improving Deep Learning Interpretability by Saliency Guided Trainingcited by 68

50 most cited of 73 All 73 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 3; 2023: 8; 2024: 23; 2025: 8; 2026: 6.

48 papers.

Sort:

  1. 2023Explainable Artificial Intelligence (XAI) 2.0: A Manifesto of Open Challenges and Interdisciplinary Research Directionscited by 123
  2. 2024Explainable Artificial Intelligence: A Survey of Needs, Techniques, Applications, and Future Directioncited by 46
  3. 2024Position: Explain to Question not to Justifycited by 20
  4. 2023Reveal to Revise: An Explainable AI Life Cycle for Iterative Bias Correction of Deep Modelscited by 17
  5. 2024Establishing and Evaluating Trustworthy AI: Overview and Research Challengescited by 12
  6. 2022Optimizing Explanations by Network Canonization and Hyperparameter Searchcited by 11
  7. 2022Explainable AI for Bioinformatics: Methods, Tools, and Applicationscited by 11
  8. 2025A Close Look at Decomposition-based XAI-Methods for Transformer Language Modelscited by 10
  9. 2023Concept Distillation: Leveraging Human-Centered Explanations for Model Improvementcited by 9
  10. 2024A survey on Concept-based Approaches For Model Improvementcited by 8
  11. 2023Strategies to exploit XAI to improve classification systemscited by 7
  12. 2024XEdgeAI: A Human-centered Industrial Inspection Framework with Data-centric Explainable Edge AI Approachcited by 7
36 more
  1. 2025Towards Desiderata-Driven Design of Visual Counterfactual Explainerscited by 7
  2. 2024Controllable Continual Test-Time Adaptationcited by 6
  3. 2023Explainable artificial intelligence approaches for brain-computer interfaces: a review and design spacecited by 5
  4. 2024The FIX Benchmark: Extracting Features Interpretable to eXpertscited by 5
  5. 2024Explainable Artificial Intelligence (XAI) for Malware Analysis: A Survey of Techniques, Applications, and Open Challengescited by 5
  6. 2024LibraGrad: Balancing Gradient Flow for Universally Better Vision Transformer Attributionscited by 5
  7. 2024Cross- and Intra-image Prototypical Learning for Multi-label Disease Diagnosis and Interpretationcited by 4
  8. 2023Reliability Scores from Saliency Map Clusters for Improved Image-based Harvest-Readiness Prediction in Cauliflowercited by 3
  9. 2024Iterative Graph Neural Network Enhancement via Frequent Subgraph Mining of Explanationscited by 3
  10. 2024X-SHIELD: Regularization for eXplainable Artificial Intelligencecited by 3
  11. 2024xAI-Drop: Don't Use What You Cannot Explaincited by 3
  12. 2024Beyond explaining: XAI-based Adaptive Learning with SHAP Clustering for Energy Consumption Predictioncited by 2
  13. 2024Explainable Interface for Human-Autonomy Teaming: A Surveycited by 2
  14. 2024Locally Testing Model Detections for Semantic Global Conceptscited by 2
  15. 2025A Survey on Interpretability in Visual Recognitioncited by 2
  16. 2024XAI-Enhanced Semantic Segmentation Models for Visual Quality Inspectioncited by 1
  17. 2024Quantifying Spatial Domain Explanations in BCI using Earth Mover's Distancecited by 1
  18. 2024Learning Visually Grounded Domain Ontologies via Embodied Conversation and Explanationcited by 1
  19. 2025Guidelines For The Choice Of The Baseline in XAI Attribution Methodscited by 1
  20. 2026Generative Explainability for Next-Generation Networks: LLM-Augmented XAI with Mutual Feature Interactionscited by 1
  21. 2022Harnessing the Power of Explanations for Incremental Training: A LIME-Based Approachcited by 0
  22. 2023On the Effectiveness of Adversarial Samples against Ensemble Learning-based Windows PE Malware Detectorscited by 0
  23. 2023Relevance-guided Neural Machine Translationcited by 0
  24. 2024Towards a general framework for improving the performance of classifiers using XAI methodscited by 0
  25. 2024Influence based explainability of brain tumors segmentation in multimodal Magnetic Resonance Imagingcited by 0
  26. 2024Model Guidance via Explanations Turns Image Classifiers into Segmentation Modelscited by 0
  27. 2024The Impact of an XAI-Augmented Approach on Binary Classification with Scarce Datacited by 0
  28. 2025IMPACTX: improving model performance by appropriately constraining the training with teacher explanationscited by 0
  29. 2025Automated Processing of eXplainable Artificial Intelligence Outputs in Deep Learning Models for Fault Diagnostics of Large Infrastructurescited by 0
  30. 2025Mind the XAI Gap: A Human-Centered LLM Framework for Democratizing Explainable AIcited by 0
  31. 2025Spatial Information Bottleneck for Interpretable Visual Recognitioncited by 0
  32. 2026Structural Compactness as a Complementary Criterion for Explanation Qualitycited by 0
  33. 2026Explaining Neural Networks in Preference Learning: a Post-hoc Inductive Logic Programming Approachcited by 0
  34. 2026Self-Explainability in Self-Adaptive and Self-Organising Systems: Status and Research Directionscited by 0
  35. 2026Explaining Temporal Graph Neural Networks via Feature-induced Information Flowcited by 0
  36. 2026ResLRP: The Role of Residual Cancellation in Attribution Instability in Vision Transformerscited by 0