mlsanityData to 7 Oct 2026

Shared Interest: Measuring Human-AI Alignment to Identify Recurring Patterns in Model Behavior

arXiv 2107.09234 ↗First posted 20 Jul 2021cs.LGcited by 29

Not on the map. Why?

Abstract (arXiv)

Saliency methods -- techniques to identify the importance of input features on a model's output -- are a common step in understanding neural network behavior. However, interpreting saliency requires tedious manual inspection to identify and aggregate patterns in model behavior, resulting in ad hoc or cherry-picked analysis. To address these concerns, we present Shared Interest: metrics for comparing model reasoning (via saliency) to human reasoning (via ground truth annotations). By providing quantitative descriptors, Shared Interest enables ranking, sorting, and aggregating inputs, thereby facilitating large-scale systematic analysis of model behavior. We use Shared Interest to identify eight recurring patterns in model behavior, such as cases where contextual features or a subset of ground truth features are most important to the model. Working with representative real-world users, we show how Shared Interest can be used to decide if a model is trustworthy, uncover issues missed in manual analyses, and enable interactive probing.

Key facts. Cited by 29 papers in the map (data through 7 Oct 2026). Builds on 37 of its 51 references that are in the map. First posted on arXiv 20 Jul 2021.

Builds on

37 of 51 references.

References · 37

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  4. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
  5. 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
  6. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
  7. 2017Axiomatic Attribution for Deep Networkscited by 3,350
  8. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,348
  9. 2013Visualizing and Understanding Convolutional Networkscited by 2,278
  10. 2019A Survey on Bias and Fairness in Machine Learningcited by 1,911
  11. 2017Towards A Rigorous Science of Interpretable Machine Learningcited by 1,400
  12. 2017Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)cited by 1,309
25 more
  1. 2018Sanity Checks for Saliency Mapscited by 1,149
  2. 2017SmoothGrad: removing noise by adding noisecited by 1,097
  3. 2014Striving for Simplicity: The All Convolutional Netcited by 983
  4. 2017Network Dissection: Quantifying Interpretability of Deep Visual Representationscited by 703
  5. 2019Skin Lesion Analysis Toward Melanoma Detection 2018: A Challenge Hosted by the International Skin Imaging Collaboration (ISIC)cited by 657
  6. 2017Skin Lesion Analysis Toward Melanoma Detection: A Challenge at the 2017 International Symposium on Biomedical Imaging (ISBI), Hosted by the International Skin Imaging Collaboration (ISIC)cited by 549
  7. 2020Captum: A unified and generic model interpretability library for PyTorchcited by 542
  8. 2020Noise or Signal: The Role of Image Backgrounds in Object Recognitioncited by 342
  9. 2017The (Un)reliability of saliency methodscited by 331
  10. 2020Are we done with ImageNet?cited by 329
  11. 2016Rationalizing Neural Predictionscited by 281
  12. 2020Large image datasets: A pyrrhic win for computer vision?cited by 237
  13. 2019The What-If Tool: Interactive Probing of Machine Learning Modelscited by 207
  14. 2020Debugging Tests for Model Explanationscited by 146
  15. 2019Sanity Checks for Saliency Metricscited by 119
  16. 2018Visual Analytics in Deep Learning: An Interrogative Survey for the Next Frontierscited by 117
  17. 2020From ImageNet to Image Classification: Contextualizing Progress on Benchmarkscited by 90
  18. 2019Benchmarking Attribution Methods with Relative Feature Importancecited by 89
  19. 2019Summit: Scaling Deep Learning Interpretability by Visualizing Activation and Attribution Summarizationscited by 72
  20. 2017ActiVis: Visual Exploration of Industry-Scale Deep Neural Network Modelscited by 56
  21. 2018GAN Lab: Understanding Complex Deep Generative Models using Interactive Visual Experimentationcited by 40
  22. 2019exBERT: A Visual Analysis Tool to Explore Learned Representations in Transformers Modelscited by 33
  23. 2018What made you do this? Understanding black-box decisions with sufficient input subsetscited by 31
  24. 2020Overinterpretation reveals image classification model pathologiescited by 28
  25. 2016A Characterization of Prediction Errorscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 4; 2023: 5; 2024: 13; 2025: 4; 2026: 2.

29 papers.

Sort:

By area:

  1. 2024Towards Human-AI Deliberation: Design and Evaluation of LLM-Empowered Deliberative AI for AI-Assisted Decision-Makingcited by 38
  2. 2024LeGrad: An Explainability Method for Vision Transformers via Feature Formation Sensitivitycited by 32
  3. 2023Selective Explanations: Leveraging Human Input to Align Explainable AIcited by 29
  4. 2024AI Can Be Cognitively Biased: An Exploratory Study on Threshold Priming in LLM-Based Batch Relevance Assessmentcited by 20
  5. 2024Position: Towards Bidirectional Human-AI Alignmentcited by 15
  6. 2023Do Large Language Models Pay Similar Attention Like Human Programmers When Generating Code?cited by 13
  7. 2024An Evaluation of Estimative Uncertainty in Large Language Modelscited by 13
  8. 2022Saliency Cards: A Framework to Characterize and Compare Saliency Methodscited by 11
  9. 2025Probing the Visualization Literacy of Vision Language Models: the Good, the Bad, and the Uglycited by 9
  10. 2022A.I. Robustness: a Human-Centered Perspective on Technological Challenges and Opportunitiescited by 8
  11. 2024Clarify: Improving Model Robustness With Natural Language Correctionscited by 8
  12. 2024JailbreakLens: Visual Analysis of Jailbreak Attacks Against Large Language Modelscited by 7
17 more
  1. 2024Abstraction Alignment: Comparing Model-Learned and Human-Encoded Conceptual Relationshipscited by 7
  2. 2021Where do Models go Wrong? Parameter-Space Saliency Maps for Explainabilitycited by 6
  3. 2023Saliency-Bench: A Comprehensive Benchmark for Evaluating Visual Explanationscited by 6
  4. 2024Allowing humans to interactively guide machines where to look does not always improve human-AI team's classification accuracycited by 5
  5. 2023Diagrammatization and Abduction to Improve AI Interpretability With Domain-Aligned Explanations for Medical Diagnosiscited by 4
  6. 2025Interpretable Decision-Making for End-to-End Autonomous Drivingcited by 4
  7. 2023CommonsenseVIS: Visualizing and Understanding Commonsense Reasoning Capabilities of Natural Language Modelscited by 2
  8. 2024TouchAI: Exploring human-AI perceptual alignment in touch through language model representationscited by 2
  9. 2025A Survey on Interpretability in Visual Recognitioncited by 2
  10. 2022A Case Report On The "A.I. Locked-In Problem": social concerns with modern NLPcited by 1
  11. 2022An Interactive Interpretability System for Breast Cancer Screening with Deep Learningcited by 1
  12. 2024POEM: Interactive Prompt Optimization for Enhancing Multimodal Reasoning of Large Language Modelscited by 1
  13. 2025Tell Me Without Telling Me: Two-Way Prediction of Visualization Literacy and Visual Attentioncited by 1
  14. 2024Dual-Channel Reliable Breast Ultrasound Image Classification Based on Explainable Attribution and Uncertainty Quantificationcited by 0
  15. 2024Graphical Perception of Saliency-based Model Explanationscited by 0
  16. 2026Rules or Weights? Comparing User Understanding of Explainable AI Techniques with the Cognitive XAI-Adaptive Modelcited by 0
  17. 2026SketchXplain: Intuitive Visual Explanations of Image Classifiers with Sketchescited by 0