mlsanityData to 14 Oct 2026

Explainable AI (XAI): A Systematic Meta-Survey of Current Challenges and Future Opportunities

arXiv 2111.06420 ↗First posted 11 Nov 2021cs.LG cs.AIcited by 125

Not on the map. Why?

Abstract (arXiv)

The past decade has seen significant progress in artificial intelligence (AI), which has resulted in algorithms being adopted for resolving a variety of problems. However, this success has been met by increasing model complexity and employing black-box AI models that lack transparency. In response to this need, Explainable AI (XAI) has been proposed to make AI more transparent and thus advance the adoption of AI in critical domains. Although there are several reviews of XAI topics in the literature that identified challenges and potential research directions in XAI, these challenges and research directions are scattered. This study, hence, presents a systematic meta-survey for challenges and future research directions in XAI organized in two themes: (1) general challenges and research directions in XAI and (2) challenges and research directions in XAI based on machine learning life cycle's phases: design, development, and deployment. We believe that our meta-survey contributes to XAI literature by providing a guide for future exploration in the XAI area.

Key facts. Cited by 125 papers in the map (data through 14 Oct 2026). Builds on 66 of its 155 references that are in the map. First posted on arXiv 11 Nov 2021.

Builds on

66 of 155 references.

References · 66

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  4. 2016Densely Connected Convolutional Networkscited by 6,720
  5. 2013Intriguing properties of neural networkscited by 5,155
  6. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,070
  7. 2013Visualizing and Understanding Convolutional Networkscited by 2,290
  8. 2019A Survey on Bias and Fairness in Machine Learningcited by 1,917
  9. 2019Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AIcited by 1,552
  10. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,475
  11. 2017Towards A Rigorous Science of Interpretable Machine Learningcited by 1,416
  12. 2017Explanation in Artificial Intelligence: Insights from the Social Sciencescited by 1,206
38 more
  1. 2018A Survey Of Methods For Explaining Black Box Modelscited by 973
  2. 2014Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Imagescited by 776
  3. 2016The Mythos of Model Interpretabilitycited by 721
  4. 2019Fooling LIME and SHAP: Adversarial Attacks on Post hoc Explanation Methodscited by 443
  5. 2019A Survey on Explainable Artificial Intelligence (XAI): Towards Medical XAIcited by 404
  6. 2020Explainability in Graph Neural Networks: A Taxonomic Surveycited by 384
  7. 2016European Union regulations on algorithmic decision-making and a "right to explanation"cited by 344
  8. 2019Interpretable machine learning: definitions, methods, and applicationscited by 305
  9. 2018Knockoff Nets: Stealing Functionality of Black-Box Modelscited by 292
  10. 2020A Survey on Neural Network Interpretabilitycited by 291
  11. 2020A Survey on the Explainability of Supervised Machine Learningcited by 227
  12. 2018Explainable Recommendation: A Survey and New Perspectivescited by 210
  13. 2015Doctor AI: Predicting Clinical Events via Recurrent Neural Networkscited by 208
  14. 2017Explainable Artificial Intelligence: Understanding, Visualizing and Interpreting Deep Learning Modelscited by 206
  15. 2020A Survey of the State of Explainable AI for Natural Language Processingcited by 206
  16. 2018Multimodal Explanations: Justifying Decisions and Pointing to the Evidencecited by 179
  17. 2020A Survey of Evaluation Metrics Used for NLG Systemscited by 164
  18. 2021Post-hoc Interpretability for Neural NLP: A Surveycited by 156
  19. 2018Visual Interpretability for Deep Learning: a Surveycited by 155
  20. 2021Explainable artificial intelligence (XAI) in deep learning-based medical image analysiscited by 147
  21. 2020Explainable Deep Learning: A Field Guide for the Uninitiatedcited by 127
  22. 2020Interpretable Machine Learning -- A Brief History, State-of-the-Art and Challengescited by 120
  23. 2019Towards Explainable Artificial Intelligencecited by 106
  24. 2020Principles and Practice of Explainable Machine Learningcited by 99
  25. 2019Analysis of Explainers of Black Box Deep Neural Networks for Computer Vision: A Surveycited by 93
  26. 2020The role of explainability in creating trustworthy artificial intelligence for health care: a comprehensive survey of the terminology, design choices, and evaluation strategiescited by 92
  27. 2020On Interpretability of Artificial Neural Networks: A Surveycited by 88
  28. 2020Causal Interpretability for Machine Learning -- Problems, Methods and Evaluationcited by 82
  29. 2020Explainable Artificial Intelligence: a Systematic Reviewcited by 79
  30. 2019Explanation in Human-AI Systems: A Literature Meta-Review, Synopsis of Key Ideas and Publications, and Bibliography for Explainable AIcited by 63
  31. 2020Efficient nonparametric statistical inference on population feature importance using Shapley valuescited by 61
  32. 2016Towards Bayesian Deep Learning: A Framework and Some Existing Methodscited by 58
  33. 2016"What is Relevant in a Text Document?": An Interpretable Machine Learning Approachcited by 53
  34. 2018Explanation Methods in Deep Learning: Users, Values, Concerns and Challengescited by 49
  35. 2017Visual Feature Attribution using Wasserstein GANscited by 47
  36. 2021A Review on Explainability in Multimodal Deep Neural Netscited by 46
  37. 2020A systematic review and taxonomy of explanations in decision support and recommender systemscited by 45
  38. 2017Explainable Planningcited by 40

50 most cited of 66 All 66 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 1; 2022: 3; 2023: 23; 2024: 53; 2025: 25; 2026: 20.

125 papers.

Sort:

By area:

  1. 2023A Survey on LLM-Generated Text Detection: Necessity, Methods, and Future Directionscited by 105
  2. 2021Explainable Artificial Intelligence for Autonomous Driving: A Comprehensive Overview and Field Guide for Future Research Directionscited by 73
  3. 2023Learning to Intervene on Concept Bottleneckscited by 29
  4. 2023A review of ensemble learning and data augmentation models for class imbalanced problems: combination, implementation and evaluationcited by 25
  5. 2023Multimodal Explainable Artificial Intelligence: A Comprehensive Review of Methodological Advances and Future Research Directionscited by 25
  6. 2024Interpretable Concept Bottlenecks to Align Reinforcement Learning Agentscited by 23
  7. 2023A Comprehensive Survey on Relation Extraction: Recent Advances and New Frontierscited by 22
  8. 2024Explainable Generative AI (GenXAI): A Survey, Conceptualization, and Research Agendacited by 22
  9. 2024XAI for All: Can Large Language Models Simplify Explainable AI?cited by 21
  10. 2024Position: Explain to Question not to Justifycited by 20
  11. 2024The Interpretable and Effective Graph Neural Additive Networkscited by 20
  12. 2023Directive Explanations for Monitoring the Risk of Diabetes Onset: Introducing Directive Data-Centric Explanations and Combinations to Support What-If Explorationscited by 18
38 more
  1. 2024Interpretable and Editable Programmatic Tree Policies for Reinforcement Learningcited by 18
  2. 2024Concept-Based Explanations in Computer Vision: Where Are We and Where Could We Go?cited by 18
  3. 2025A Comprehensive Survey on Self-Interpretable Neural Networkscited by 18
  4. 2023Explaining Quantum Circuits with Shapley Values: Towards Explainable Quantum Machine Learningcited by 17
  5. 2023Learning by Self-Explainingcited by 17
  6. 2024Graph Neural Networks for Job Shop Scheduling Problems: A Surveycited by 15
  7. 2022Deletion and Insertion Tests in Regression Modelscited by 14
  8. 2023Multi-Modal Deep Learning for Credit Rating Prediction Using Text and Numerical Data Streamscited by 13
  9. 2023FINER: Enhancing State-of-the-art Classifiers with Feature Attribution to Facilitate Security Analysiscited by 12
  10. 2023Achieving Diversity in Counterfactual Explanations: a Review and Discussioncited by 11
  11. 2024How should AI decisions be explained? Requirements for Explanations from the Perspective of European Lawcited by 11
  12. 2024HackAtari: Atari Learning Environments for Robust and Continual Reinforcement Learningcited by 11
  13. 2024Intrinsic User-Centric Interpretability through Global Mixture of Expertscited by 9
  14. 2024A Review of Multimodal Explainable Artificial Intelligence: Past, Present and Futurecited by 9
  15. 2025CoE: Chain-of-Explanation via Automatic Visual Concept Circuit Description and Polysemanticity Quantificationcited by 9
  16. 2025Which LIME should I trust? Concepts, Challenges, and Solutionscited by 8
  17. 2022Requirements Engineering for Machine Learning: A Review and Reflectioncited by 7
  18. 2024Safety Implications of Explainable Artificial Intelligence in End-to-End Autonomous Drivingcited by 7
  19. 2024Layer-Wise Relevance Propagation with Conservation Property for ResNetcited by 7
  20. 2024Beyond Model Interpretability: Socio-Structural Explanations in Machine Learningcited by 6
  21. 2024Interplay between Federated Learning and Explainable Artificial Intelligence: a Scoping Reviewcited by 6
  22. 2023Explainable artificial intelligence approaches for brain-computer interfaces: a review and design spacecited by 5
  23. 2024The FIX Benchmark: Extracting Features Interpretable to eXpertscited by 5
  24. 2024Explainable Artificial Intelligence (XAI) for Malware Analysis: A Survey of Techniques, Applications, and Open Challengescited by 5
  25. 2024Combining Embedding-Based and Semantic-Based Models for Post-hoc Explanations in Recommender Systemscited by 4
  26. 2024False Sense of Security in Explainable Artificial Intelligence (XAI)cited by 4
  27. 2024Adapting Physics-Informed Neural Networks to Improve ODE Optimization in Mosquito Population Dynamicscited by 4
  28. 2025Explainable AI for Clinical Outcome Prediction: A Survey of Clinician Perceptions and Preferencescited by 4
  29. 2025Fewer Than 1% of Explainable AI Papers Validate Explainability with Humanscited by 4
  30. 2023On the Relationship Between Interpretability and Explainability in Machine Learningcited by 3
  31. 2024ADESSE: Advice Explanations in Complex Repeated Decision-Making Environmentscited by 3
  32. 2024Towards consistency of rule-based explainer and black box model -- fusion of rule induction and XAI-based feature importancecited by 3
  33. 2024The Interaction Layer: An Exploration for Co-Designing User-LLM Interactions in Parental Wellbeing Support Systemscited by 3
  34. 2024Multi-SpaCE: Multi-Objective Subsequence-based Sparse Counterfactual Explanations for Multivariate Time Series Classificationcited by 3
  35. 2025On the Effectiveness of Methods and Metrics for Explainable AI in Remote Sensing Image Scene Classificationcited by 3
  36. 2025A Geometric Unification of Concept Learning with Concept Conescited by 3
  37. 2023A Brief Review of Explainable Artificial Intelligence in Healthcarecited by 2
  38. 2023Evolutionary approaches to explainable machine learningcited by 2

50 most cited of 125 All 125 (JSON)