mlsanityData to 14 Oct 2026

The Thousand Faces of Explainable AI Along the Machine Learning Life Cycle: Industrial Reality and Current State of Research

arXiv 2310.07882 ↗First posted 11 Oct 2023cs.LG cs.AI cs.HCcited by 4

Not on the map. Why?

Abstract (arXiv)

In this paper, we investigate the practical relevance of explainable artificial intelligence (XAI) with a special focus on the producing industries and relate them to the current state of academic XAI research. Our findings are based on an extensive series of interviews regarding the role and applicability of XAI along the Machine Learning (ML) lifecycle in current industrial practice and its expected relevance in the future. The interviews were conducted among a great variety of roles and key stakeholders from different industry sectors. On top of that, we outline the state of XAI research by providing a concise review of the relevant literature. This enables us to provide an encompassing overview covering the opinions of the surveyed persons as well as the current state of academic research. By comparing our interview results with the current research approaches we reveal several discrepancies. While a multitude of different XAI approaches exists, most of them are centered around the model evaluation phase and data scientists. Their versatile capabilities for other stages are currently either not sufficiently explored or not popular among practitioners. In line with existing work, our findings also confirm that more efforts are needed to enable also non-expert users' interpretation and understanding of opaque AI models with existing methods and frameworks.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 76 of its 126 references that are in the map. First posted on arXiv 11 Oct 2023.

Builds on

76 of 126 references.

References · 76

  1. 2014Explaining and Harnessing Adversarial Examplescited by 7,364
  2. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,733
  3. 2017A Unified Approach to Interpreting Model Predictionscited by 6,555
  4. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,070
  5. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,371
  6. 2017Understanding Black-box Predictions via Influence Functionscited by 1,741
  7. 2017Towards A Rigorous Science of Interpretable Machine Learningcited by 1,416
  8. 2017Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)cited by 1,319
  9. 2016Adversarial Machine Learning at Scalecited by 936
  10. 2019Data Shapley: Equitable Valuation of Data for Machine Learningcited by 629
  11. 2020Transformer Interpretability Beyond Attention Visualizationcited by 627
  12. 2017One pixel attack for fooling deep neural networkscited by 582
38 more
  1. 2020Estimating Training Data Influence by Tracing Gradient Descentcited by 580
  2. 2020What Neural Networks Memorize and Why: Discovering the Long Tail via Influence Estimationcited by 497
  3. 2017Right for the Right Reasons: Training Differentiable Models by Constraining their Explanationscited by 294
  4. 2018Consistent Individualized Feature Attribution for Tree Ensemblescited by 285
  5. 2019Towards Efficient Data Valuation Based on the Shapley Valuecited by 281
  6. 2018Improving fairness in machine learning systems: What do industry practitioners need?cited by 262
  7. 2020Opportunities and Challenges in Explainable Artificial Intelligence (XAI): A Surveycited by 254
  8. 2020Influence Functions in Deep Learning Are Fragilecited by 252
  9. 2018Learning to Explain: An Information-Theoretic Perspective on Model Interpretationcited by 248
  10. 2019InterpretML: A Unified Framework for Machine Learning Interpretabilitycited by 246
  11. 2017Improving the Adversarial Robustness and Interpretability of Deep Neural Networks by Regularizing their Input Gradientscited by 236
  12. 2017Distilling a Neural Network Into a Soft Decision Treecited by 236
  13. 2019Explainable Machine Learning in Deploymentcited by 211
  14. 2018Representer Point Selection for Explaining Deep Neural Networkscited by 187
  15. 2016Synthesizing the preferred inputs for neurons in neural networks via deep generator networkscited by 181
  16. 2018Local Rule-Based Explanations of Black Box Decision Systemscited by 166
  17. 2022The Disagreement Problem in Explainable Machine Learning: A Practitioner's Perspectivecited by 159
  18. 2020Improving KernelSHAP: Practical Shapley Value Estimation via Linear Regressioncited by 154
  19. 2019Efficient Task-Specific Data Valuation for Nearest Neighbor Algorithmscited by 148
  20. 2019On the Accuracy of Influence Functions for Measuring Group Effectscited by 147
  21. 2019Interpretations are useful: penalizing explanations to align neural networks with prior knowledgecited by 145
  22. 2022If Influence Functions are the Answer, Then What is the Question?cited by 144
  23. 2020Making deep neural networks right for the right scientific reasons by interacting with their explanationscited by 142
  24. 2021Benchmarking and Survey of Explanation Methods for Black Box Modelscited by 136
  25. 2021FastSHAP: Real-Time Shapley Value Estimationcited by 125
  26. 2019Explaining Deep Neural Networks with a Polynomial Time Algorithm for Shapley Values Approximationcited by 115
  27. 2019Attention in Natural Language Processingcited by 101
  28. 2018L-Shapley and C-Shapley: Efficient Model Interpretation for Structured Datacited by 100
  29. 2020Estimating Example Difficulty Using Variance of Gradientscited by 97
  30. 2020Right for the Right Concept: Revising Neuro-Symbolic Concepts by Interacting with their Explanationscited by 94
  31. 2020On the Tractability of SHAP Explanationscited by 92
  32. 2019Pruning by Explaining: A Novel Criterion for Deep Neural Network Pruningcited by 89
  33. 2020A Distributional Framework for Data Valuationcited by 80
  34. 2019On Second-Order Group Influence Functions for Black-Box Predictionscited by 79
  35. 2021Input Similarity from the Neural Network Perspectivecited by 79
  36. 2021Explainable Artificial Intelligence (XAI) on TimeSeries Data: A Surveycited by 72
  37. 2019Incorporating Priors with Feature Attribution on Text Classificationcited by 68
  38. 2019Data Cleansing for Models Trained with SGDcited by 67

50 most cited of 76 All 76 (JSON)

Built on by

Cited by year

Citing papers per year: 2024: 1; 2025: 3.

4 papers.

  1. 2025Improving Perturbation-based Explanations by Understanding the Role of Uncertainty Calibrationcited by 3
  2. 2025MoRE-LLM: Mixture of Rule Experts Guided by a Large Language Modelcited by 1
  3. 2024Root Causing Prediction Anomalies Using Explainable AIcited by 0
  4. 2025LVLM-Aided Alignment of Task-Specific Vision Modelscited by 0