# Dissonance Between Human and Machine Understanding

arXiv: https://arxiv.org/abs/2101.07337 · HTML: https://mlsanity.com/paper/2101.07337/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 75 references that are in the map.
- First posted on arXiv 18 Jan 2021.

## Builds on (most cited first, 18 of 18)

- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [Understanding Black-box Predictions via Influence Functions](https://mlsanity.com/paper/1703.04730/) (2017)
- [Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning](https://mlsanity.com/paper/1602.07261/) (2016)
- [Show, Attend and Tell: Neural Image Caption Generation with Visual Attention](https://mlsanity.com/paper/1502.03044/) (2015)
- [Towards A Rigorous Science of Interpretable Machine Learning](https://mlsanity.com/paper/1702.08608/) (2017)
- [The Mythos of Model Interpretability](https://mlsanity.com/paper/1606.03490/) (2016)
- [Right for the Right Reasons: Training Differentiable Models by Constraining their Explanations](https://mlsanity.com/paper/1703.03717/) (2017)
- [Generalisation in humans and deep neural networks](https://mlsanity.com/paper/1808.08750/) (2018)
- [Model-Agnostic Interpretability of Machine Learning](https://mlsanity.com/paper/1606.05386/) (2016)
- [Interpretable classifiers using rules and Bayesian analysis: Building a better stroke prediction model](https://mlsanity.com/paper/1511.01644/) (2015)
- [Building machines that adapt and compute like brains](https://mlsanity.com/paper/1711.04203/) (2017)
- ['It's Reducing a Human Being to a Percentage'; Perceptions of Justice in Algorithmic Decisions](https://mlsanity.com/paper/1801.10408/) (2018)
- [Embracing Error to Enable Rapid Crowdsourcing](https://mlsanity.com/paper/1602.04506/) (2016)
- [Learning Credible Models](https://mlsanity.com/paper/1711.03190/) (2017)

## Built on by (most cited first, 18 of 18)

- [Zorro: Valid, Sparse, and Stable Explanations in Graph Neural Networks](https://mlsanity.com/paper/2105.08621/) (2021)
- [ConvXAI: Delivering Heterogeneous AI Explanations via Conversations to Support Human-AI Scientific Writing](https://mlsanity.com/paper/2305.09770/) (2023)
- [AutoGPart: Intermediate Supervision Search for Generalizable 3D Part Segmentation](https://mlsanity.com/paper/2203.06558/) (2022)
- [Why is plausibility surprisingly problematic as an XAI criterion?](https://mlsanity.com/paper/2303.17707/) (2023)
- [Learnt Sparsification for Interpretable Graph Neural Networks](https://mlsanity.com/paper/2106.12920/) (2021)
- [Toward a Unified Framework for Debugging Concept-based Models](https://mlsanity.com/paper/2109.11160/) (2021)
- [FaxPlainAC: A Fact-Checking Tool Based on EXPLAINable Models with HumAn Correction in the Loop](https://mlsanity.com/paper/2110.10144/) (2021)
- [Two-Stage Holistic and Contrastive Explanation of Image Classification](https://mlsanity.com/paper/2306.06339/) (2023)
- [Do humans and Convolutional Neural Networks attend to similar areas during scene classification: Effects of task and image type](https://mlsanity.com/paper/2307.13345/) (2023)
- [The Need for Interpretable Features: Motivation and Taxonomy](https://mlsanity.com/paper/2202.11748/) (2022)
- [Designing a Direct Feedback Loop between Humans and Convolutional Neural Networks through Local Explanations](https://mlsanity.com/paper/2307.04036/) (2023)
- [Perception of Visual Content: Differences Between Humans and Foundation Models](https://mlsanity.com/paper/2411.18968/) (2024)
- [Human-AI Collaborative Game Testing with Vision Language Models](https://mlsanity.com/paper/2501.11782/) (2025)
- [Going PLACES: Participatory Localized Red Teaming for Text-to-Image Safety in the Global South](https://mlsanity.com/paper/2605.19190/) (2026)
- [Debiased-CAM to mitigate systematic error with faithful visual explanations of machine learning](https://mlsanity.com/paper/2201.12835/) (2022)
- [Behavior Matters: An Alternative Perspective on Promoting Responsible Data Science](https://mlsanity.com/paper/2410.17273/) (2024)
- [How Can One Choose the Best CAM-Based Explainability Method for a CNN Model?](https://mlsanity.com/paper/2605.02007/) (2026)
- [CLARK: Closed-loop Learning for Adaptive Reasoning over Knowledge Graphs](https://mlsanity.com/paper/2607.19996/) (2026)

## Abstract (arXiv)

Complex machine learning models are deployed in several critical domains including healthcare and autonomous vehicles nowadays, albeit as functional black boxes. Consequently, there has been a recent surge in interpreting decisions of such complex models in order to explain their actions to humans. Models that correspond to human interpretation of a task are more desirable in certain contexts and can help attribute liability, build trust, expose biases and in turn build better models. It is, therefore, crucial to understand how and which models conform to human understanding of tasks. In this paper, we present a large-scale crowdsourcing study that reveals and quantifies the dissonance between human and machine understanding, through the lens of an image classification task. In particular, we seek to answer the following questions: Which (well-performing) complex ML models are closer to humans in their use of features to make accurate predictions? How does task difficulty affect the feature selection capability of machines in comparison to humans? Are humans consistently better at selecting features that make image recognition more accurate? Our findings have important implications on human-machine collaboration, considering that a long term goal in the field of artificial intelligence is to make machines capable of learning and reasoning like humans.