mlsanityData to 7 Oct 2026

Explainable Deep Learning: A Field Guide for the Uninitiated

arXiv 2004.14545 ↗First posted 30 Apr 2020cs.LG cs.AI stat.MLcited by 127

Not on the map. Why?

Abstract (arXiv)

Deep neural networks (DNNs) have become a proven and indispensable machine learning tool. As a black-box model, it remains difficult to diagnose what aspects of the model's input drive the decisions of a DNN. In countless real-world domains, from legislation and law enforcement to healthcare, such diagnosis is essential to ensure that DNN decisions are driven by aspects appropriate in the context of its use. The development of methods and studies enabling the explanation of a DNN's decisions has thus blossomed into an active, broad area of research. A practitioner wanting to study explainable deep learning may be intimidated by the plethora of orthogonal directions the field has taken. This complexity is further exacerbated by competing definitions of what it means ``to explain'' the actions of a DNN and to evaluate an approach's ``ability to explain''. This article offers a field guide to explore the space of explainable deep learning aimed at those uninitiated in the field. The field guide: i) Introduces three simple dimensions defining the space of foundational methods that contribute to explainable deep learning, ii) discusses the evaluations for model explanations, iii) places explainability in the context of other related deep learning research areas, and iv) finally elaborates on user-oriented explanation designing and potential future directions on explainable deep learning. We hope the guide is used as an easy-to-digest starting point for those just embarking on research in this field.

Key facts. Cited by 127 papers in the map (data through 7 Oct 2026). First posted on arXiv 30 Apr 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 27; 2022: 32; 2023: 28; 2024: 25; 2025: 10; 2026: 5.

127 papers.

Sort:

By area:

  1. 2022Improving alignment of dialogue agents via targeted human judgementscited by 527
  2. 2022From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AIcited by 299
  3. 2022Teaching language models to support answers with verified quotescited by 290
  4. 2021Explainable AI (XAI): A Systematic Meta-Survey of Current Challenges and Future Opportunitiescited by 125
  5. 2021A Comprehensive Taxonomy for Explainable Artificial Intelligence: A Systematic Survey of Surveys on Methods and Conceptscited by 90
  6. 2021Explainability of deep vision-based autonomous driving systems: Review and challengescited by 80
  7. 2021ILDC for CJPE: Indian Legal Documents Corpus for Court Judgment Prediction and Explanationcited by 78
  8. 2022Leveraging Explanations in Interactive Machine Learning: An Overviewcited by 52
  9. 2022Logic-Based Explainability in Machine Learningcited by 47
  10. 2021A Review on Explainability in Multimodal Deep Neural Netscited by 46
  11. 2022On Tackling Explanation Redundancy in Decision Treescited by 44
  12. 2021On the proper role of linguistically-oriented deep net analysis in linguistic theorizingcited by 43
38 more
  1. 2021Tell me why! Explanations support learning relational and causal structurecited by 36
  2. 2022A Survey on Graph Counterfactual Explanations: Definitions, Methods, Evaluation, and Research Challengescited by 30
  3. 2023Artificial intelligence to advance Earth observation: : A review of models, recent trends, and pathways forwardcited by 28
  4. 2021Interpretable agent communication from scratch (with a generic visual processor emerging on the side)cited by 25
  5. 2024Interpretable Concept Bottlenecks to Align Reinforcement Learning Agentscited by 23
  6. 2022Fairness and Explainability: Bridging the Gap Towards Fair Model Explanationscited by 22
  7. 2024Interpretable and Editable Programmatic Tree Policies for Reinforcement Learningcited by 18
  8. 2025Security and Quality in LLM-Generated Code: A Multi-Language, Multi-Model Analysiscited by 18
  9. 2022Astronomia ex machina: a history, primer, and outlook on neural networks in astronomycited by 17
  10. 2023Learning by Self-Explainingcited by 17
  11. 2021SurvNAM: The machine learning survival model explanationcited by 16
  12. 2021XAI Handbook: Towards a Unified Framework for Explainable AIcited by 14
  13. 2022Survey on Fair Reinforcement Learning: Theory and Practicecited by 14
  14. 2023From Neural Activations to Concepts: A Survey on Explaining Concepts in Neural Networkscited by 13
  15. 2021Explainable Artificial Intelligence for Human Decision-Support System in Medical Domaincited by 12
  16. 2022What You See is What You Classify: Black Box Attributionscited by 12
  17. 2024Hindsight PRIORs for Reward Learning from Human Preferencescited by 12
  18. 2022Explainability in reinforcement learning: perspective and positioncited by 11
  19. 2022Greybox XAI: a Neural-Symbolic learning framework to produce interpretable predictions for image classificationcited by 11
  20. 2023Enhancing Human-like Multi-Modal Reasoning: A New Challenging Dataset and Comprehensive Frameworkcited by 11
  21. 2024HackAtari: Atari Learning Environments for Robust and Continual Reinforcement Learningcited by 11
  22. 2021When and How to Fool Explainable Models (and Humans) with Adversarial Examplescited by 10
  23. 2022Limitations of Deep Learning for Inverse Problems on Digital Hardwarecited by 9
  24. 2022Constrained Parameter Inference as a Principle for Learningcited by 9
  25. 2021Ensembles of Random SHAPscited by 8
  26. 2021Nested Multiple Instance Learning with Attention Mechanismscited by 7
  27. 2024Efficient Deep Learning with Decorrelated Backpropagationcited by 7
  28. 2024ECG Unveiled: Analysis of Client Re-identification Risks in Real-World ECG Datasetscited by 7
  29. 2021An Imprecise SHAP as a Tool for Explaining the Class Probability Distributions under Limited Training Datacited by 6
  30. 2022Gradient Backpropagation based Feature Attribution to Enable Explainable-AI on the Edgecited by 6
  31. 2023Improving Interpretability via Explicit Word Interaction Graph Layercited by 6
  32. 2023SAR-UNet: Small Attention Residual UNet for Explainable Nowcasting Taskscited by 6
  33. 2023Interpretable Mesomorphic Networks for Tabular Datacited by 6
  34. 2021Exploring Explainable AI in the Financial Sector: Perspectives of Banks and Supervisory Authoritiescited by 5
  35. 2022Explainable Misinformation Detection Across Multiple Social Media Platformscited by 5
  36. 2023FederatedTrust: A Solution for Trustworthy Federated Learningcited by 5
  37. 2023Non-Uniform Interpolation in Integrated Gradients for Low-Latency Explainable-AIcited by 5
  38. 2023Explainable AI applications in the Medical Domain: a systematic reviewcited by 5

50 most cited of 127 All 127 (JSON)