# Explanation Bias is a Product: Revealing the Hidden Lexical and Position Preferences in Post-Hoc Feature Attribution

arXiv: https://arxiv.org/abs/2512.11108 · HTML: https://mlsanity.com/paper/2512.11108/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 26 of its 41 references that are in the map.
- First posted on arXiv 11 Dec 2025.

## Builds on (most cited first, 26 of 26)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [Efficient Streaming Language Models with Attention Sinks](https://mlsanity.com/paper/2309.17453/) (2023)
- [Learning Important Features Through Propagating Activation Differences](https://mlsanity.com/paper/1704.02685/) (2017)
- [DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharing](https://mlsanity.com/paper/2111.09543/) (2021)
- [Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inference](https://mlsanity.com/paper/2412.13663/) (2024)
- [ERASER: A Benchmark to Evaluate Rationalized NLP Models](https://mlsanity.com/paper/1911.03429/) (2019)
- [Rationalizing Neural Predictions](https://mlsanity.com/paper/1606.04155/) (2016)
- [A Diagnostic Study of Explainability Techniques for Text Classification](https://mlsanity.com/paper/2009.13295/) (2020)
- [Towards Faithful Model Explanation in NLP: A Survey](https://mlsanity.com/paper/2209.11326/) (2022)
- [Rethinking Cooperative Rationalization: Introspective Extraction and Complement Control](https://mlsanity.com/paper/1910.13294/) (2019)
- [The Out-of-Distribution Problem in Explainability and Search Methods for Feature Importance Explanations](https://mlsanity.com/paper/2106.00786/) (2021)
- [Look at the First Sentence: Position Bias in Question Answering](https://mlsanity.com/paper/2004.14602/) (2020)
- [ferret: a Framework for Benchmarking Explainers on Transformers](https://mlsanity.com/paper/2208.01575/) (2022)
- [Automatic Extraction of Causal Relations from Natural Language Texts: A Comprehensive Survey](https://mlsanity.com/paper/1605.07895/) (2016)
- [Towards Interpreting BERT for Reading Comprehension Based QA](https://mlsanity.com/paper/2010.08983/) (2020)
- [Many Faces of Feature Importance: Comparing Built-in and Post-hoc Feature Importance in Text Classification](https://mlsanity.com/paper/1910.08534/) (2019)
- [Short-circuiting Shortcuts: Mechanistic Investigation of Shortcuts in Text Classification](https://mlsanity.com/paper/2505.06032/) (2025)
- [Neighboring Words Affect Human Interpretation of Saliency Explanations](https://mlsanity.com/paper/2305.02679/) (2023)
- [Dynamic Top-k Estimation Consolidates Disagreement between Feature Attribution Methods](https://mlsanity.com/paper/2310.05619/) (2023)
- [The Effect of Model Size on LLM Post-hoc Explainability via LIME](https://mlsanity.com/paper/2405.05348/) (2024)
- [The Role of Syntactic Span Preferences in Post-Hoc Explanation Disagreement](https://mlsanity.com/paper/2403.19424/) (2024)

## Abstract (arXiv)

Good quality explanations strengthen the understanding of language models and data. Feature attribution methods, such as Integrated Gradient, are a type of post-hoc explainer that can provide token-level insights. However, explanations on the same input may vary greatly due to underlying biases of different methods. Users may be aware of this issue and mistrust their utility, while unaware users may trust them inadequately. In this work, we delve beyond the superficial inconsistencies between attribution methods, structuring their biases through a model- and method-agnostic framework of three evaluation metrics. We systematically assess both lexical and position bias (what and where in the input) for two transformers; first, in a controlled, pseudo-random classification task on artificial data; then, in a semi-controlled causal relation detection task on natural data. We find a trade-off between lexical and position biases in our model comparison, with models that score high on one type score low on the other. We also find signs that anomalous explanations are more likely to be biased.