# Fixing confirmation bias in feature attribution methods via semantic match

arXiv: https://arxiv.org/abs/2307.00897 · HTML: https://mlsanity.com/paper/2307.00897/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 64 references that are in the map.
- First posted on arXiv 3 Jul 2023.

## Builds on (most cited first, 30 of 38)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Visualizing and Understanding Convolutional Networks](https://mlsanity.com/paper/1311.2901/) (2013)
- [Towards A Rigorous Science of Interpretable Machine Learning](https://mlsanity.com/paper/1702.08608/) (2017)
- [Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)](https://mlsanity.com/paper/1711.11279/) (2017)
- [Sanity Checks for Saliency Maps](https://mlsanity.com/paper/1810.03292/) (2018)
- [Striving for Simplicity: The All Convolutional Net](https://mlsanity.com/paper/1412.6806/) (2014)
- [Probing Classifiers: Promises, Shortcomings, and Advances](https://mlsanity.com/paper/2102.12452/) (2021)
- [The Mythos of Model Interpretability](https://mlsanity.com/paper/1606.03490/) (2016)
- [Towards Faithfully Interpretable NLP Systems: How should we define and evaluate faithfulness?](https://mlsanity.com/paper/2004.03685/) (2020)
- [Explaining Explanations: An Overview of Interpretability of Machine Learning](https://mlsanity.com/paper/1806.00069/) (2018)
- [Interpretation of Neural Networks is Fragile](https://mlsanity.com/paper/1710.10547/) (2017)
- [Fooling LIME and SHAP: Adversarial Attacks on Post hoc Explanation Methods](https://mlsanity.com/paper/1911.02508/) (2019)
- [The (Un)reliability of saliency methods](https://mlsanity.com/paper/1711.00867/) (2017)
- [Post-hoc Concept Bottleneck Models](https://mlsanity.com/paper/2205.15480/) (2022)
- [Model-Agnostic Interpretability of Machine Learning](https://mlsanity.com/paper/1606.05386/) (2016)
- [Explainable Machine Learning in Deployment](https://mlsanity.com/paper/1909.06342/) (2019)
- [Explainable deep learning models in medical image analysis](https://mlsanity.com/paper/2005.13799/) (2020)
- [Do Feature Attribution Methods Correctly Attribute Features?](https://mlsanity.com/paper/2104.14403/) (2021)
- [Look at the First Sentence: Position Bias in Question Answering](https://mlsanity.com/paper/2004.14602/) (2020)
- [What Does Explainable AI Really Mean? A New Conceptualization of Perspectives](https://mlsanity.com/paper/1710.00794/) (2017)
- [Entropy-based Logic Explanations of Neural Networks](https://mlsanity.com/paper/2106.06804/) (2021)
- [This Looks Like That, Because ... Explaining Prototypes for Interpretable Image Recognition](https://mlsanity.com/paper/2011.02863/) (2020)
- [Robust Explainability: A Tutorial on Gradient-Based Attribution Methods for Deep Neural Networks](https://mlsanity.com/paper/2107.11400/) (2021)
- [Local Explanations via Necessity and Sufficiency: Unifying Theory and Practice](https://mlsanity.com/paper/2103.14651/) (2021)
- [On Baselines for Local Feature Attributions](https://mlsanity.com/paper/2101.00905/) (2021)

## Built on by (most cited first, 2 of 2)

- [Interpretable phenotyping of Heart Failure patients with Dutch discharge letters](https://mlsanity.com/paper/2505.24619/) (2025)
- [Is my model perplexed for the right reason? Contrasting LLMs' Benchmark Behavior with Token-Level Perplexity](https://mlsanity.com/paper/2603.29396/) (2026)

## Abstract (arXiv)

Feature attribution methods have become a staple method to disentangle the complex behavior of black box models. Despite their success, some scholars have argued that such methods suffer from a serious flaw: they do not allow a reliable interpretation in terms of human concepts. Simply put, visualizing an array of feature contributions is not enough for humans to conclude something about a model's internal representations, and confirmation bias can trick users into false beliefs about model behavior. We argue that a structured approach is required to test whether our hypotheses on the model are confirmed by the feature attributions. This is what we call the "semantic match" between human concepts and (sub-symbolic) explanations. Building on the conceptual framework put forward in Cin`a et al. [2023], we propose a structured approach to evaluate semantic match in practice. We showcase the procedure in a suite of experiments spanning tabular and image data, and show how the assessment of semantic match can give insight into both desirable (e.g., focusing on an object relevant for prediction) and undesirable model behaviors (e.g., focusing on a spurious correlation). We couple our experimental results with an analysis on the metrics to measure semantic match, and argue that this approach constitutes the first step towards resolving the issue of confirmation bias in XAI.