mlsanityData to 14 Oct 2026

Leveraging Explanations in Interactive Machine Learning: An Overview

arXiv 2207.14526 ↗First posted 29 Jul 2022cs.LGcited by 53

Not on the map. Why?

Abstract (arXiv)

Explanations have gained an increasing level of interest in the AI and Machine Learning (ML) communities in order to improve model transparency and allow users to form a mental model of a trained ML model. However, explanations can go beyond this one way communication as a mechanism to elicit user control, because once users understand, they can then provide feedback. The goal of this paper is to present an overview of research where explanations are combined with interactive capabilities as a mean to learn new models from scratch and to edit and debug existing ones. To this end, we draw a conceptual map of the state-of-the-art, grouping relevant approaches based on their intended purpose and on how they structure the interaction, highlighting similarities and differences between them. We also discuss open research issues and outline possible directions forward, with the hope of spurring further research on this blooming research topic.

Key facts. Cited by 53 papers in the map (data through 14 Oct 2026). Builds on 121 of its 210 references that are in the map. First posted on arXiv 29 Jul 2022.

Builds on

121 of 210 references.

References · 121

  1. 2017Attention Is All You Needcited by 60,361
  2. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,733
  3. 2017A Unified Approach to Interpreting Model Predictionscited by 6,555
  4. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  5. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,070
  6. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,548
  7. 2017Axiomatic Attribution for Deep Networkscited by 3,384
  8. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,371
  9. 2020Shortcut Learning in Deep Neural Networkscited by 1,969
  10. 2017Understanding Black-box Predictions via Influence Functionscited by 1,741
  11. 2017Explanation in Artificial Intelligence: Insights from the Social Sciencescited by 1,206
  12. 2018Sanity Checks for Saliency Mapscited by 1,156
38 more
  1. 2017Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPRcited by 1,089
  2. 2020Concept Bottleneck Modelscited by 1,032
  3. 2018A Survey Of Methods For Explaining Black Box Modelscited by 973
  4. 2018This Looks Like That: Deep Learning for Interpretable Image Recognitioncited by 737
  5. 2018Explaining Explanations: An Overview of Interpretability of Machine Learningcited by 508
  6. 2018Towards Robust Interpretability with Self-Explaining Neural Networkscited by 505
  7. 2018A Benchmark for Interpretability Methods in Deep Neural Networkscited by 476
  8. 2018e-SNLI: Natural Language Inference with Natural Language Explanationscited by 468
  9. 2019Learning the Difference that Makes a Difference with Counterfactually-Augmented Datacited by 439
  10. 2021Causal Abstractions of Neural Networkscited by 422
  11. 2017Snorkel: Rapid Training Data Creation with Weak Supervisioncited by 372
  12. 2021Interpretable Machine Learning: Fundamental Principles and 10 Grand Challengescited by 367
  13. 2019Unmasking Clever Hans Predictors and Assessing What Machines Really Learncited by 365
  14. 2017Methods for Interpreting and Understanding Deep Neural Networkscited by 345
  15. 2017The (Un)reliability of saliency methodscited by 333
  16. 2020Performative Predictioncited by 327
  17. 2018Neural-Symbolic VQA: Disentangling Reasoning from Vision and Language Understandingcited by 304
  18. 2017Right for the Right Reasons: Training Differentiable Models by Constraining their Explanationscited by 294
  19. 2020Influence Functions in Deep Learning Are Fragilecited by 252
  20. 2020Algorithmic Recourse: from Counterfactual Explanations to Interventionscited by 250
  21. 2020Concept Whitening for Interpretable Image Recognitioncited by 225
  22. 2021Polyjuice: Generating Counterfactuals for Explaining, Evaluating, and Improving Modelscited by 222
  23. 2019Explanations can be manipulated and geometry is to blamecited by 216
  24. 2020Neural Prototype Trees for Interpretable Fine-grained Image Recognitioncited by 214
  25. 2018Explainable Recommendation: A Survey and New Perspectivescited by 210
  26. 2021Towards Causal Representation Learningcited by 209
  27. 2019Informed Machine Learning -- A Taxonomy and Survey of Integrating Knowledge into Learning Systemscited by 205
  28. 2020Problems with Shapley-value-based explanations as feature importance measurescited by 200
  29. 2018Representer Point Selection for Explaining Deep Neural Networkscited by 187
  30. 2018Local Rule-Based Explanations of Black Box Decision Systemscited by 166
  31. 2019Neural-Symbolic Computing: An Effective Methodology for Principled Integration of Machine Learning and Reasoningcited by 151
  32. 2020Debugging Tests for Model Explanationscited by 147
  33. 2019Interpretations are useful: penalizing explanations to align neural networks with prior knowledgecited by 145
  34. 2020Making deep neural networks right for the right scientific reasons by interacting with their explanationscited by 142
  35. 2018A Symbolic Approach to Explaining Bayesian Network Classifierscited by 131
  36. 2021Promises and Pitfalls of Black-Box Concept Learning Modelscited by 131
  37. 2019Taking a HINT: Leveraging Explanations to Make Vision and Language Models More Groundedcited by 128
  38. 2020Explainable Deep Learning: A Field Guide for the Uninitiatedcited by 127

50 most cited of 121 All 121 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 1; 2023: 16; 2024: 16; 2025: 12; 2026: 8.

53 papers.

Sort:

By area:

  1. 2023Discover and Cure: Concept-aware Mitigation of Spurious Correlationcited by 71
  2. 2023Not All Neuro-Symbolic Concepts Are Created Equal: Analysis and Mitigation of Reasoning Shortcutscited by 63
  3. 2024BEARS Make Neuro-Symbolic Models Aware of their Reasoning Shortcutscited by 37
  4. 2024Interactive AI with Retrieval-Augmented Generation for Next Generation Networkingcited by 34
  5. 2023Neuro-Symbolic Continual Learning: Knowledge, Reasoning Shortcuts and Concept Rehearsalcited by 33
  6. 2023Learning to Intervene on Concept Bottleneckscited by 29
  7. 2023Interactive Explanations by Conflict Resolution via Argumentative Exchangescited by 24
  8. 2024Interpretable Concept Bottlenecks to Align Reinforcement Learning Agentscited by 23
  9. 2023Interpretability is in the Mind of the Beholder: A Causal Framework for Human-interpretable Representation Learningcited by 22
  10. 2023Learning by Self-Explainingcited by 17
  11. 2023Studying How to Efficiently and Effectively Guide Models with Explanationscited by 16
  12. 2024Reconsidering Faithfulness in Regular, Self-Explainable and Domain Invariant GNNscited by 16
38 more
  1. 2025Prompting in the Dark: Assessing Human Performance in Prompt Engineering for Data Labeling When Gold Labels Are Absentcited by 15
  2. 2025If Concept Bottlenecks are the Question, are Foundation Models the Answer?cited by 15
  3. 2024EXMOS: Explanatory Model Steering Through Multifaceted Explanations and Data Configurationscited by 14
  4. 2025Shortcuts and Identifiability in Concept-based Models from a Neuro-Symbolic Lenscited by 13
  5. 2024HackAtari: Atari Learning Environments for Robust and Continual Reinforcement Learningcited by 11
  6. 2025Beyond Topological Self-Explainable GNNs: A Formal Explainability Perspectivecited by 11
  7. 2024A survey on Concept-based Approaches For Model Improvementcited by 8
  8. 2024Unpacking Human-AI interactions: From interaction primitives to a design spacecited by 7
  9. 2025Improving Human Verification of LLM Reasoning through Interactive Explanation Interfacescited by 7
  10. 2023How Faithful are Self-Explainable GNNs?cited by 5
  11. 2026GNN Explanations that do not Explain and How to find Themcited by 5
  12. 2023Lessons Learned from EXMOS User Studies: A Technical Report Summarizing Key Takeaways from User Studies Conducted to Evaluate The EXMOS Platformcited by 4
  13. 2023The Thousand Faces of Explainable AI Along the Machine Learning Life Cycle: Industrial Reality and Current State of Researchcited by 4
  14. 2024Learning To Guide Human Decision Makers With Vision-Language Modelscited by 4
  15. 2024An Explanatory Model Steering System for Collaboration between Domain Experts and AIcited by 4
  16. 2022Model free variable importance for high dimensional datacited by 3
  17. 2023One Explanation Does Not Fit XILcited by 2
  18. 2024Explainable Interface for Human-Autonomy Teaming: A Surveycited by 2
  19. 2025Knowing What You Cannot Explain: Learning to Reject Low-Quality Explanationscited by 2
  20. 2025AIM: Amending Inherent Interpretability via Self-Supervised Maskingcited by 2
  21. 2023Increasing Performance And Sample Efficiency With Model-agnostic Interactive Feature Attributionscited by 1
  22. 2023Learning from Exemplary Explanationscited by 1
  23. 2024Interactive Explainable Anomaly Detection for Industrial Settingscited by 1
  24. 2024Learning Visually Grounded Domain Ontologies via Embodied Conversation and Explanationcited by 1
  25. 2025Exploring the Impact of Explainable AI and Cognitive Capabilities on Users' Decisionscited by 1
  26. 2025Activation Matching for Explanation Generationcited by 1
  27. 2026Do Sparse Autoencoders Learn Meaningful Concept Hierarchies?cited by 1
  28. 2023MaskSearch: Querying Image Masks at Scalecited by 0
  29. 2023Evaluating how interactive visualizations can assist in finding samples where and how computer vision models make mistakescited by 0
  30. 2024Demonstration of MaskSearch: Efficiently Querying Image Masks for Machine Learning Workflowscited by 0
  31. 2024Towards a Research Community in Interpretable Reinforcement Learning: the InterpPol Workshopcited by 0
  32. 2024Spurious Correlations in Concept Drift: Can Explanatory Interaction Help?cited by 0
  33. 2025Chat-CBM: Towards Interactive Concept Bottleneck Models with Frozen Large Language Modelscited by 0
  34. 2025EXP-CAM: Explanation Generation and Circuit Discovery Using Classifier Activation Matchingcited by 0
  35. 2025Human Cognitive Biases in Explanation-Based Interaction: The Case of Within and Between Session Order Effectcited by 0
  36. 2026Explanatory Interactive Machine Learning for Bias Mitigation in Visual Gender Classificationcited by 0
  37. 2026Position: Explainable AI is Causality in Disguisecited by 0
  38. 2026Ceci n'est pas une explication: Evaluating Explanation Failures as Explainability Pitfalls in Language Learning Systemscited by 0

50 most cited of 53 All 53 (JSON)