# Support-Contra Asymmetry in LLM Explanations

arXiv: https://arxiv.org/abs/2510.21884 · HTML: https://mlsanity.com/paper/2510.21884/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 35 references that are in the map.
- First posted on arXiv 23 Oct 2025.

## Builds on (most cited first, 27 of 27)

- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Towards A Rigorous Science of Interpretable Machine Learning](https://mlsanity.com/paper/1702.08608/) (2017)
- [Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting](https://mlsanity.com/paper/2305.04388/) (2023)
- [Towards Faithfully Interpretable NLP Systems: How should we define and evaluate faithfulness?](https://mlsanity.com/paper/2004.03685/) (2020)
- [ERASER: A Benchmark to Evaluate Rationalized NLP Models](https://mlsanity.com/paper/1911.03429/) (2019)
- [A Diagnostic Study of Explainability Techniques for Text Classification](https://mlsanity.com/paper/2009.13295/) (2020)
- [Towards Faithful Model Explanation in NLP: A Survey](https://mlsanity.com/paper/2209.11326/) (2022)
- [Faithfulness Tests for Natural Language Explanations](https://mlsanity.com/paper/2305.18029/) (2023)
- [Faithfulness vs. Plausibility: On the (Un)Reliability of Explanations from Large Language Models](https://mlsanity.com/paper/2402.04614/) (2024)
- [Are self-explanations from Large Language Models faithful?](https://mlsanity.com/paper/2401.07927/) (2024)
- [The Out-of-Distribution Problem in Explainability and Search Methods for Feature Importance Explanations](https://mlsanity.com/paper/2106.00786/) (2021)
- [Do Models Explain Themselves? Counterfactual Simulatability of Natural Language Explanations](https://mlsanity.com/paper/2307.08678/) (2023)
- [On Measuring Faithfulness or Self-consistency of Natural Language Explanations](https://mlsanity.com/paper/2311.07466/) (2023)
- [Shapley Flow: A Graph-based Approach to Interpreting Model Predictions](https://mlsanity.com/paper/2010.14592/) (2020)
- [Measuring Chain of Thought Faithfulness by Unlearning Reasoning Steps](https://mlsanity.com/paper/2502.14829/) (2025)
- [Towards Consistent Natural-Language Explanations via Explanation-Consistency Finetuning](https://mlsanity.com/paper/2401.13986/) (2024)
- [Comparing Explanation Methods for Traditional Machine Learning Models Part 1: An Overview of Current Methods and Quantifying Their Disagreement](https://mlsanity.com/paper/2211.08943/) (2022)
- [Incorporating Attribution Importance for Improving Faithfulness Metrics](https://mlsanity.com/paper/2305.10496/) (2023)
- [Evaluating Human Alignment and Model Faithfulness of LLM Rationale](https://mlsanity.com/paper/2407.00219/) (2024)
- [A Causal Lens for Evaluating Faithfulness Metrics](https://mlsanity.com/paper/2502.18848/) (2025)
- [Many Faces of Feature Importance: Comparing Built-in and Post-hoc Feature Importance in Text Classification](https://mlsanity.com/paper/1910.08534/) (2019)
- [Can LLMs Explain Themselves Counterfactually?](https://mlsanity.com/paper/2502.18156/) (2025)
- [Self-Critique and Refinement for Faithful Natural Language Explanations](https://mlsanity.com/paper/2505.22823/) (2025)
- [From Feature Importance to Natural Language Explanations Using LLMs with RAG](https://mlsanity.com/paper/2407.20990/) (2024)
- [Aligning What LLMs Do and Say: Towards Self-Consistent Explanations](https://mlsanity.com/paper/2506.07523/) (2025)
- [A Necessary Step toward Faithfulness: Measuring and Improving Consistency in Free-Text Explanations](https://mlsanity.com/paper/2505.19299/) (2025)

## Built on by (most cited first, 1 of 1)

- [From Performance to Purpose: A Sociotechnical Taxonomy for Evaluating Large Language Model Utility](https://mlsanity.com/paper/2602.20513/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) increasingly produce natural language explanations alongside their predictions, yet it remains unclear whether these explanations reference predictive cues present in the input text. In this work, we present an empirical study of how LLM-generated explanations align with predictive lexical evidence from an external model in text classification tasks. To analyze this relationship, we compare explanation content against interpretable feature importance signals extracted from transparent linear classifiers. These reference models allow us to partition predictive lexical cues into supporting and contradicting evidence relative to the predicted label. Across three benchmark datasets-WIKIONTOLOGY, AG NEWS, and IMDB-we observe a consistent empirical pattern that we term support-contra asymmetry. Explanations accompanying correct predictions tend to reference more supporting lexical cues and fewer contradicting cues, whereas explanations associated with incorrect predictions reference substantially more contradicting evidence. This pattern appears consistently across datasets, across reference model families (logistic regression and linear SVM), and across multiple feature retrieval depths. These results suggest that LLM explanations often reflect lexical signals that are predictive for the task when predictions are correct, while incorrect predictions are more frequently associated with explanations that reference misleading cues present in the input. Our findings provide a simple empirical perspective on explanation-evidence alignment and illustrate how external sources of predictive evidence can be used to analyze the behavior of LLM-generated explanations.