mlsanityData to 7 Oct 2026

Making deep neural networks right for the right scientific reasons by interacting with their explanations

arXiv 2001.05371 ↗First posted 15 Jan 2020cs.LG cs.AI stat.MLcited by 141

Not on the map. Why?

Abstract (arXiv)

Deep neural networks have shown excellent performances in many real-world applications. Unfortunately, they may show "Clever Hans"-like behavior -- making use of confounding factors within datasets -- to achieve high performance. In this work, we introduce the novel learning setting of "explanatory interactive learning" (XIL) and illustrate its benefits on a plant phenotyping research task. XIL adds the scientist into the training loop such that she interactively revises the original model via providing feedback on its explanations. Our experimental results demonstrate that XIL can help avoiding Clever Hans moments in machine learning and encourages (or discourages, if appropriate) trust into the underlying model.

Key facts. Cited by 141 papers in the map (data through 7 Oct 2026). First posted on arXiv 15 Jan 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 22; 2022: 22; 2023: 36; 2024: 36; 2025: 18; 2026: 7.

141 papers.

Sort:

By area:

  1. 2021Interpretable Machine Learning: Fundamental Principles and 10 Grand Challengescited by 364
  2. 2022Can language models learn from explanations in context?cited by 248
  3. 2021Large Pre-trained Language Models Contain Human-like Biases of What is Right and Wrong to Docited by 178
  4. 2022From Attribution Maps to Human-Understandable Explanations through Concept Relevance Propagationcited by 141
  5. 2021EnD: Entangling and Disentangling deep representations for bias correctioncited by 105
  6. 2022A Consistent and Efficient Evaluation Strategy for Attribution Methodscited by 100
  7. 2023Tell Your Model Where to Attend: Post-hoc Attention Steering for LLMscited by 82
  8. 2021IAIA-BL: A Case-based Interpretable Deep Learning Model for Classification of Mass Lesions in Digital Mammographycited by 72
  9. 2023Not All Neuro-Symbolic Concepts Are Created Equal: Analysis and Mitigation of Reasoning Shortcutscited by 63
  10. 2021Explanation-Based Human Debugging of NLP Models: A Surveycited by 53
  11. 2022Concept-level Debugging of Part-Prototype Networkscited by 52
  12. 2022Leveraging Explanations in Interactive Machine Learning: An Overviewcited by 52
38 more
  1. 2021Interactive Label Cleaning with Example-based Explanationscited by 48
  2. 2022Beyond Explaining: Opportunities and Challenges of XAI-Based Model Improvementcited by 48
  3. 2021A Review on Explainability in Multimodal Deep Neural Netscited by 46
  4. 2023Knowing About Knowing: An Illusion of Human Competence Can Hinder Appropriate Reliance on AI Systemscited by 40
  5. 2024Enhancing Descriptive Image Quality Assessment with A Large-scale Multi-modal Datasetcited by 38
  6. 2021Tell me why! Explanations support learning relational and causal structurecited by 36
  7. 2021Fast Axiomatic Attribution for Neural Networkscited by 35
  8. 2022Going Beyond XAI: A Systematic Survey for Explanation-Guided Learningcited by 35
  9. 2023OCAtari: Object-Centric Atari 2600 Reinforcement Learning Environmentscited by 30
  10. 2023Learning to Intervene on Concept Bottleneckscited by 29
  11. 2023Towards Trustable Skin Cancer Diagnosis via Rewriting Model's Decisioncited by 27
  12. 2023An Empirical Evaluation of the Rashomon Effect in Explainable Machine Learningcited by 25
  13. 2021Knowledge-Grounded Self-Rationalization via Extractive and Natural Language Explanationscited by 24
  14. 2021Interactive Disentanglement: Learning Concepts by Interacting with their Prototype Representationscited by 24
  15. 2023ICICLE: Interpretable Class Incremental Continual Learningcited by 24
  16. 2024The Clever Hans Effect in Unsupervised Learningcited by 24
  17. 2024Interpretable Concept Bottlenecks to Align Reinforcement Learning Agentscited by 23
  18. 2024Explainable Generative AI (GenXAI): A Survey, Conceptualization, and Research Agendacited by 22
  19. 2023Understanding the (Extra-)Ordinary: Validating Deep Model Decisions with Prototypical Concept-based Explanationscited by 21
  20. 2023From Hope to Safety: Unlearning Biases of Deep Models via Gradient Penalization in Latent Spacecited by 20
  21. 2024Pix2Code: Learning to Compose Neural Visual Concepts as Programscited by 18
  22. 2024Interpretable and Editable Programmatic Tree Policies for Reinforcement Learningcited by 18
  23. 2022Revealing Hidden Context Bias in Segmentation and Object Detection through Concept-specific Explanationscited by 17
  24. 2023Reveal to Revise: An Explainable AI Life Cycle for Iterative Bias Correction of Deep Modelscited by 17
  25. 2023Learning by Self-Explainingcited by 17
  26. 2023Studying How to Efficiently and Effectively Guide Models with Explanationscited by 16
  27. 2021Explainable Machine Learning with Prior Knowledge: An Overviewcited by 14
  28. 2021Toward Learning Human-aligned Cross-domain Robust Models by Countering Misaligned Featurescited by 14
  29. 2022DORA: Exploring Outlier Representations in Deep Neural Networkscited by 14
  30. 2023Towards Fixing Clever-Hans Predictors with Counterfactual Knowledge Distillationcited by 14
  31. 2024EXMOS: Explanatory Model Steering Through Multifaceted Explanations and Data Configurationscited by 14
  32. 2024Neural Concept Bindercited by 14
  33. 2022Self-explaining deep models with logic rule reasoningcited by 13
  34. 2024HackAtari: Atari Learning Environments for Robust and Continual Reinforcement Learningcited by 11
  35. 2024Problem Solving Through Human-AI Preference-Based Cooperationcited by 11
  36. 2021When and How to Fool Explainable Models (and Humans) with Adversarial Examplescited by 10
  37. 2022Identifying Spurious Correlations and Correcting them with an Explanation-based Learningcited by 10
  38. 2024Regulating Model Reliance on Non-Robust Features by Smoothing Input Marginal Densitycited by 10

50 most cited of 141 All 141 (JSON)