# Goodhart's Law Applies to NLP's Explanation Benchmarks

arXiv: https://arxiv.org/abs/2308.14272 · HTML: https://mlsanity.com/paper/2308.14272/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 37 references that are in the map.
- First posted on arXiv 28 Aug 2023.

## Builds on (most cited first, 30 of 32)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions](https://mlsanity.com/paper/1905.10044/) (2019)
- [Show, Attend and Tell: Neural Image Caption Generation with Visual Attention](https://mlsanity.com/paper/1502.03044/) (2015)
- [FEVER: a large-scale dataset for Fact Extraction and VERification](https://mlsanity.com/paper/1803.05355/) (2018)
- [RISE: Randomized Input Sampling for Explanation of Black-box Models](https://mlsanity.com/paper/1806.07421/) (2018)
- [The Mythos of Model Interpretability](https://mlsanity.com/paper/1606.03490/) (2016)
- [Towards Faithfully Interpretable NLP Systems: How should we define and evaluate faithfulness?](https://mlsanity.com/paper/2004.03685/) (2020)
- [Captum: A unified and generic model interpretability library for PyTorch](https://mlsanity.com/paper/2009.07896/) (2020)
- [ERASER: A Benchmark to Evaluate Rationalized NLP Models](https://mlsanity.com/paper/1911.03429/) (2019)
- [A Benchmark for Interpretability Methods in Deep Neural Networks](https://mlsanity.com/paper/1806.10758/) (2018)
- [Interpretation of Neural Networks is Fragile](https://mlsanity.com/paper/1710.10547/) (2017)
- [Fooling LIME and SHAP: Adversarial Attacks on Post hoc Explanation Methods](https://mlsanity.com/paper/1911.02508/) (2019)
- [Evaluating the visualization of what a Deep Neural Network has learned](https://mlsanity.com/paper/1509.06321/) (2015)
- [Is Attention Interpretable?](https://mlsanity.com/paper/1906.03731/) (2019)
- [Explaining by Removing: A Unified Framework for Model Explanation](https://mlsanity.com/paper/2011.14878/) (2020)
- [Explanations can be manipulated and geometry is to blame](https://mlsanity.com/paper/1906.07983/) (2019)
- [The Disagreement Problem in Explainable Machine Learning: A Practitioner's Perspective](https://mlsanity.com/paper/2202.01602/) (2022)
- [OpenXAI: Towards a Transparent Evaluation of Model Explanations](https://mlsanity.com/paper/2206.11104/) (2022)
- [Fooling Neural Network Interpretations via Adversarial Model Manipulation](https://mlsanity.com/paper/1902.02041/) (2019)
- [Learning to Deceive with Attention-Based Explanations](https://mlsanity.com/paper/1909.07913/) (2019)
- [Rethinking Cooperative Rationalization: Introspective Extraction and Complement Control](https://mlsanity.com/paper/1910.13294/) (2019)
- [Evaluating Explanations: How much do explanations from the teacher aid students?](https://mlsanity.com/paper/2012.00893/) (2020)
- [The Out-of-Distribution Problem in Explainability and Search Methods for Feature Importance Explanations](https://mlsanity.com/paper/2106.00786/) (2021)
- [Fairwashing Explanations with Off-Manifold Detergent](https://mlsanity.com/paper/2007.09969/) (2020)
- [Inferring Which Medical Treatments Work from Reports of Clinical Trials](https://mlsanity.com/paper/1904.01606/) (2019)
- [Have We Learned to Explain?: How Interpretability Methods Can Learn to Encode Predictions in their Interpretations](https://mlsanity.com/paper/2103.01890/) (2021)

## Built on by (most cited first, 10 of 10)

- [GenAI-Bench: Evaluating and Improving Compositional Text-to-Visual Generation](https://mlsanity.com/paper/2406.13743/) (2024)
- [GNN Explanations that do not Explain and How to find Them](https://mlsanity.com/paper/2601.20815/) (2026)
- [Post Turing: Mapping the landscape of LLM Evaluation](https://mlsanity.com/paper/2311.02049/) (2023)
- [Explanations that reveal all through the definition of encoding](https://mlsanity.com/paper/2411.02664/) (2024)
- [Gender Bias in Explainability: Investigating Performance Disparity in Post-hoc Methods](https://mlsanity.com/paper/2505.01198/) (2025)
- [The Strong, Weak and Benign Goodhart's law. An independence-free and paradigm-agnostic formalisation](https://mlsanity.com/paper/2505.23445/) (2025)
- [Explanation Multiplicity: Circuit-Level Interpretability Evidence Does Not Survive Defensible Analytic Variation](https://mlsanity.com/paper/2608.13754/) (2026)
- [Interpreting Black-Box Large Language Models with Sentence-Level Energy Landscapes](https://mlsanity.com/paper/2608.02879/) (2026)
- [Beyond Unsafe Detection: Counterfactually Anchored Evidence Attribution for Multi-Turn LLM Safety Failures](https://mlsanity.com/paper/2609.27773/) (2026)
- [Metric-Construction Coupling Inflates Measured Synthetic Dialect Recovery](https://mlsanity.com/paper/2610.00164/) (2026)

## Abstract (arXiv)

Despite the rising popularity of saliency-based explanations, the research community remains at an impasse, facing doubts concerning their purpose, efficacy, and tendency to contradict each other. Seeking to unite the community's efforts around common goals, several recent works have proposed evaluation metrics. In this paper, we critically examine two sets of metrics: the ERASER metrics (comprehensiveness and sufficiency) and the EVAL-X metrics, focusing our inquiry on natural language processing. First, we show that we can inflate a model's comprehensiveness and sufficiency scores dramatically without altering its predictions or explanations on in-distribution test inputs. Our strategy exploits the tendency for extracted explanations and their complements to be "out-of-support" relative to each other and in-distribution inputs. Next, we demonstrate that the EVAL-X metrics can be inflated arbitrarily by a simple method that encodes the label, even though EVAL-X is precisely motivated to address such exploits. Our results raise doubts about the ability of current metrics to guide explainability research, underscoring the need for a broader reassessment of what precisely these metrics are intended to capture.