# Impact of Feedback Type on Explanatory Interactive Learning

arXiv: https://arxiv.org/abs/2209.12476 · HTML: https://mlsanity.com/paper/2209.12476/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 17 references that are in the map.
- First posted on arXiv 26 Sep 2022.

## Builds on (most cited first, 9 of 9)

- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Concept Bottleneck Models](https://mlsanity.com/paper/2007.04612/) (2020)
- [Right for the Right Reasons: Training Differentiable Models by Constraining their Explanations](https://mlsanity.com/paper/1703.03717/) (2017)
- [Making deep neural networks right for the right scientific reasons by interacting with their explanations](https://mlsanity.com/paper/2001.05371/) (2020)
- [Taking a HINT: Leveraging Explanations to Make Vision and Language Models More Grounded](https://mlsanity.com/paper/1902.03751/) (2019)
- [Towards Teachable Reasoning Systems: Using a Dynamic Memory of User Feedback for Continual System Improvement](https://mlsanity.com/paper/2204.13074/) (2022)
- [Machine Guides, Human Supervises: Interactive Learning with Global Explanations](https://mlsanity.com/paper/2009.09723/) (2020)
- [Improving Neural Model Performance through Natural Language Feedback on Their Explanations](https://mlsanity.com/paper/2104.08765/) (2021)

## Built on by (most cited first, 7 of 7)

- [Identifying Spurious Correlations and Correcting them with an Explanation-based Learning](https://mlsanity.com/paper/2211.08285/) (2022)
- [Targeted Activation Penalties Help CNNs Ignore Spurious Signals](https://mlsanity.com/paper/2311.12813/) (2023)
- [Unlearning Spurious Correlations in Chest X-ray Classification](https://mlsanity.com/paper/2308.01119/) (2023)
- [Learning from Exemplary Explanations](https://mlsanity.com/paper/2307.06026/) (2023)
- [Distance-Aware eXplanation Based Learning](https://mlsanity.com/paper/2309.05548/) (2023)
- [Towards a general framework for improving the performance of classifiers using XAI methods](https://mlsanity.com/paper/2403.10373/) (2024)
- [IMPACTX: improving model performance by appropriately constraining the training with teacher explanations](https://mlsanity.com/paper/2502.12222/) (2025)

## Abstract (arXiv)

Explanatory Interactive Learning (XIL) collects user feedback on visual model explanations to implement a Human-in-the-Loop (HITL) based interactive learning scenario. Different user feedback types will have different impacts on user experience and the cost associated with collecting feedback since different feedback types involve different levels of image annotation. Although XIL has been used to improve classification performance in multiple domains, the impact of different user feedback types on model performance and explanation accuracy is not well studied. To guide future XIL work we compare the effectiveness of two different user feedback types in image classification tasks: (1) instructing an algorithm to ignore certain spurious image features, and (2) instructing an algorithm to focus on certain valid image features. We use explanations from a Gradient-weighted Class Activation Mapping (GradCAM) based XIL model to support both feedback types. We show that identifying and annotating spurious image features that a model finds salient results in superior classification and explanation accuracy than user feedback that tells a model to focus on valid image features.