# Empirical Characterization of Rationale Stability Under Controlled Perturbations for Explainable Pattern Recognition

arXiv: https://arxiv.org/abs/2604.04456 · HTML: https://mlsanity.com/paper/2604.04456/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 28 of its 38 references that are in the map.
- First posted on arXiv 6 Apr 2026.

## Builds on (most cited first, 28 of 28)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [Counterfactual Fairness](https://mlsanity.com/paper/1703.06856/) (2017)
- [This Looks Like That: Deep Learning for Interpretable Image Recognition](https://mlsanity.com/paper/1806.10574/) (2018)
- [Fooling LIME and SHAP: Adversarial Attacks on Post hoc Explanation Methods](https://mlsanity.com/paper/1911.02508/) (2019)
- [OpenXAI: Towards a Transparent Evaluation of Model Explanations](https://mlsanity.com/paper/2206.11104/) (2022)
- [Reliable Post hoc Explanations: Modeling Uncertainty in Explainability](https://mlsanity.com/paper/2008.05030/) (2020)
- [Interpreting CNNs via Decision Trees](https://mlsanity.com/paper/1802.00121/) (2018)
- [Is Cosine-Similarity of Embeddings Really About Similarity?](https://mlsanity.com/paper/2403.05440/) (2024)
- [The effectiveness of feature attribution methods and its correlation with automatic evaluation scores](https://mlsanity.com/paper/2105.14944/) (2021)
- [Causal Interpretability for Machine Learning -- Problems, Methods and Evaluation](https://mlsanity.com/paper/2003.03934/) (2020)
- [Which Explanation Should I Choose? A Function Approximation Perspective to Characterizing Post Hoc Explanations](https://mlsanity.com/paper/2206.01254/) (2022)
- [Does Explainable Artificial Intelligence Improve Human Decision-Making?](https://mlsanity.com/paper/2006.11194/) (2020)
- [Interactive Label Cleaning with Example-based Explanations](https://mlsanity.com/paper/2106.03922/) (2021)
- [How Good is your Explanation? Algorithmic Stability Measures to Assess the Quality of Explanations for Deep Neural Networks](https://mlsanity.com/paper/2009.04521/) (2020)
- [On Locality of Local Explanation Models](https://mlsanity.com/paper/2106.14648/) (2021)
- [SEAT: Stable and Explainable Attention](https://mlsanity.com/paper/2211.13290/) (2022)
- [Consistent Explanations by Contrastive Learning](https://mlsanity.com/paper/2110.00527/) (2021)
- [F-Fidelity: A Robust Framework for Faithfulness Evaluation of Explainable AI](https://mlsanity.com/paper/2410.02970/) (2024)
- [Towards Robust Fidelity for Evaluating Explainability of Graph Neural Networks](https://mlsanity.com/paper/2310.01820/) (2023)
- [Towards Modeling Uncertainties of Self-explaining Neural Networks via Conformal Prediction](https://mlsanity.com/paper/2401.01549/) (2024)
- [Neural Reasoning Networks: Efficient Interpretable Neural Networks With Automatic Textual Explanations](https://mlsanity.com/paper/2410.07966/) (2024)
- [Maximizing Signal in Human-Model Preference Alignment](https://mlsanity.com/paper/2503.04910/) (2025)
- [Are Logistic Models Really Interpretable?](https://mlsanity.com/paper/2406.13427/) (2024)

## Abstract (arXiv)

Reliable pattern recognition systems should exhibit consistent behavior across similar inputs, and their explanations should remain stable. However, most Explainable AI evaluations remain instance centric and do not explicitly quantify whether attribution patterns are consistent across samples that share the same class or represent small variations of the same input. In this work, we propose a novel metric aimed at assessing the consistency of model explanations, ensuring that models consistently reflect the intended objectives and consistency under label-preserving perturbations. We implement this metric using a pre-trained BERT model on the SST-2 sentiment analysis dataset, with additional robustness tests on RoBERTa, DistilBERT, and IMDB, applying SHAP to compute feature importance for various test samples. The proposed metric quantifies the cosine similarity of SHAP values for inputs with the same label, aiming to detect inconsistent behaviors, such as biased reliance on certain features or failure to maintain consistent reasoning for similar predictions. Through a series of experiments, we evaluate the ability of this metric to identify misaligned predictions and inconsistencies in model explanations. These experiments are compared against standard fidelity metrics to assess whether the new metric can effectively identify when a model's behavior deviates from its intended objectives. The proposed framework provides a deeper understanding of model behavior by enabling more robust verification of rationale stability, which is critical for building trustworthy AI systems. By quantifying whether models rely on consistent attribution patterns for similar inputs, the proposed approach supports more robust evaluation of model behavior in practical pattern recognition pipelines. Our code is publicly available at https://github.com/anmspro/ESS-XAI-Stability.