# Shared Interest: Measuring Human-AI Alignment to Identify Recurring Patterns in Model Behavior

arXiv: https://arxiv.org/abs/2107.09234 · HTML: https://mlsanity.com/paper/2107.09234/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 51 references that are in the map.
- First posted on arXiv 20 Jul 2021.

## Builds on (most cited first, 30 of 37)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [Visualizing and Understanding Convolutional Networks](https://mlsanity.com/paper/1311.2901/) (2013)
- [A Survey on Bias and Fairness in Machine Learning](https://mlsanity.com/paper/1908.09635/) (2019)
- [Towards A Rigorous Science of Interpretable Machine Learning](https://mlsanity.com/paper/1702.08608/) (2017)
- [Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)](https://mlsanity.com/paper/1711.11279/) (2017)
- [Sanity Checks for Saliency Maps](https://mlsanity.com/paper/1810.03292/) (2018)
- [SmoothGrad: removing noise by adding noise](https://mlsanity.com/paper/1706.03825/) (2017)
- [Striving for Simplicity: The All Convolutional Net](https://mlsanity.com/paper/1412.6806/) (2014)
- [Network Dissection: Quantifying Interpretability of Deep Visual Representations](https://mlsanity.com/paper/1704.05796/) (2017)
- [Skin Lesion Analysis Toward Melanoma Detection 2018: A Challenge Hosted by the International Skin Imaging Collaboration (ISIC)](https://mlsanity.com/paper/1902.03368/) (2019)
- [Skin Lesion Analysis Toward Melanoma Detection: A Challenge at the 2017 International Symposium on Biomedical Imaging (ISBI), Hosted by the International Skin Imaging Collaboration (ISIC)](https://mlsanity.com/paper/1710.05006/) (2017)
- [Captum: A unified and generic model interpretability library for PyTorch](https://mlsanity.com/paper/2009.07896/) (2020)
- [Noise or Signal: The Role of Image Backgrounds in Object Recognition](https://mlsanity.com/paper/2006.09994/) (2020)
- [The (Un)reliability of saliency methods](https://mlsanity.com/paper/1711.00867/) (2017)
- [Are we done with ImageNet?](https://mlsanity.com/paper/2006.07159/) (2020)
- [Rationalizing Neural Predictions](https://mlsanity.com/paper/1606.04155/) (2016)
- [Large image datasets: A pyrrhic win for computer vision?](https://mlsanity.com/paper/2006.16923/) (2020)
- [The What-If Tool: Interactive Probing of Machine Learning Models](https://mlsanity.com/paper/1907.04135/) (2019)
- [Debugging Tests for Model Explanations](https://mlsanity.com/paper/2011.05429/) (2020)
- [Sanity Checks for Saliency Metrics](https://mlsanity.com/paper/1912.01451/) (2019)
- [Visual Analytics in Deep Learning: An Interrogative Survey for the Next Frontiers](https://mlsanity.com/paper/1801.06889/) (2018)
- [From ImageNet to Image Classification: Contextualizing Progress on Benchmarks](https://mlsanity.com/paper/2005.11295/) (2020)
- [Benchmarking Attribution Methods with Relative Feature Importance](https://mlsanity.com/paper/1907.09701/) (2019)

## Built on by (most cited first, 29 of 29)

- [Towards Human-AI Deliberation: Design and Evaluation of LLM-Empowered Deliberative AI for AI-Assisted Decision-Making](https://mlsanity.com/paper/2403.16812/) (2024)
- [LeGrad: An Explainability Method for Vision Transformers via Feature Formation Sensitivity](https://mlsanity.com/paper/2404.03214/) (2024)
- [Selective Explanations: Leveraging Human Input to Align Explainable AI](https://mlsanity.com/paper/2301.09656/) (2023)
- [AI Can Be Cognitively Biased: An Exploratory Study on Threshold Priming in LLM-Based Batch Relevance Assessment](https://mlsanity.com/paper/2409.16022/) (2024)
- [Position: Towards Bidirectional Human-AI Alignment](https://mlsanity.com/paper/2406.09264/) (2024)
- [Do Large Language Models Pay Similar Attention Like Human Programmers When Generating Code?](https://mlsanity.com/paper/2306.01220/) (2023)
- [An Evaluation of Estimative Uncertainty in Large Language Models](https://mlsanity.com/paper/2405.15185/) (2024)
- [Saliency Cards: A Framework to Characterize and Compare Saliency Methods](https://mlsanity.com/paper/2206.02958/) (2022)
- [Probing the Visualization Literacy of Vision Language Models: the Good, the Bad, and the Ugly](https://mlsanity.com/paper/2504.05445/) (2025)
- [A.I. Robustness: a Human-Centered Perspective on Technological Challenges and Opportunities](https://mlsanity.com/paper/2210.08906/) (2022)
- [Clarify: Improving Model Robustness With Natural Language Corrections](https://mlsanity.com/paper/2402.03715/) (2024)
- [JailbreakLens: Visual Analysis of Jailbreak Attacks Against Large Language Models](https://mlsanity.com/paper/2404.08793/) (2024)
- [Abstraction Alignment: Comparing Model-Learned and Human-Encoded Conceptual Relationships](https://mlsanity.com/paper/2407.12543/) (2024)
- [Where do Models go Wrong? Parameter-Space Saliency Maps for Explainability](https://mlsanity.com/paper/2108.01335/) (2021)
- [Saliency-Bench: A Comprehensive Benchmark for Evaluating Visual Explanations](https://mlsanity.com/paper/2310.08537/) (2023)
- [Allowing humans to interactively guide machines where to look does not always improve human-AI team's classification accuracy](https://mlsanity.com/paper/2404.05238/) (2024)
- [Diagrammatization and Abduction to Improve AI Interpretability With Domain-Aligned Explanations for Medical Diagnosis](https://mlsanity.com/paper/2302.01241/) (2023)
- [Interpretable Decision-Making for End-to-End Autonomous Driving](https://mlsanity.com/paper/2508.18898/) (2025)
- [CommonsenseVIS: Visualizing and Understanding Commonsense Reasoning Capabilities of Natural Language Models](https://mlsanity.com/paper/2307.12382/) (2023)
- [TouchAI: Exploring human-AI perceptual alignment in touch through language model representations](https://mlsanity.com/paper/2406.06587/) (2024)
- [A Survey on Interpretability in Visual Recognition](https://mlsanity.com/paper/2507.11099/) (2025)
- [A Case Report On The "A.I. Locked-In Problem": social concerns with modern NLP](https://mlsanity.com/paper/2209.12687/) (2022)
- [An Interactive Interpretability System for Breast Cancer Screening with Deep Learning](https://mlsanity.com/paper/2210.08979/) (2022)
- [POEM: Interactive Prompt Optimization for Enhancing Multimodal Reasoning of Large Language Models](https://mlsanity.com/paper/2406.03843/) (2024)
- [Tell Me Without Telling Me: Two-Way Prediction of Visualization Literacy and Visual Attention](https://mlsanity.com/paper/2508.03713/) (2025)
- [Dual-Channel Reliable Breast Ultrasound Image Classification Based on Explainable Attribution and Uncertainty Quantification](https://mlsanity.com/paper/2401.03664/) (2024)
- [Graphical Perception of Saliency-based Model Explanations](https://mlsanity.com/paper/2406.07702/) (2024)
- [Rules or Weights? Comparing User Understanding of Explainable AI Techniques with the Cognitive XAI-Adaptive Model](https://mlsanity.com/paper/2602.19620/) (2026)
- [SketchXplain: Intuitive Visual Explanations of Image Classifiers with Sketches](https://mlsanity.com/paper/2606.17646/) (2026)

## Abstract (arXiv)

Saliency methods -- techniques to identify the importance of input features on a model's output -- are a common step in understanding neural network behavior. However, interpreting saliency requires tedious manual inspection to identify and aggregate patterns in model behavior, resulting in ad hoc or cherry-picked analysis. To address these concerns, we present Shared Interest: metrics for comparing model reasoning (via saliency) to human reasoning (via ground truth annotations). By providing quantitative descriptors, Shared Interest enables ranking, sorting, and aggregating inputs, thereby facilitating large-scale systematic analysis of model behavior. We use Shared Interest to identify eight recurring patterns in model behavior, such as cases where contextual features or a subset of ground truth features are most important to the model. Working with representative real-world users, we show how Shared Interest can be used to decide if a model is trustworthy, uncover issues missed in manual analyses, and enable interactive probing.