# Are Data-driven Explanations Robust against Out-of-distribution Data?

arXiv: https://arxiv.org/abs/2303.16390 · HTML: https://mlsanity.com/paper/2303.16390/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 47 of its 66 references that are in the map.
- First posted on arXiv 29 Mar 2023.

## Builds on (most cited first, 30 of 47)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [Convex optimization](https://mlsanity.com/paper/2106.01946/) (2021)
- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [Invariant Risk Minimization](https://mlsanity.com/paper/1907.02893/) (2019)
- [WILDS: A Benchmark of in-the-Wild Distribution Shifts](https://mlsanity.com/paper/2012.07421/) (2020)
- [Sanity Checks for Saliency Maps](https://mlsanity.com/paper/1810.03292/) (2018)
- [In Search of Lost Domain Generalization](https://mlsanity.com/paper/2007.01434/) (2020)
- [RISE: Randomized Input Sampling for Explanation of Black-box Models](https://mlsanity.com/paper/1806.07421/) (2018)
- [Learning to Generalize: Meta-Learning for Domain Generalization](https://mlsanity.com/paper/1710.03463/) (2017)
- [Out-of-Distribution Generalization via Risk Extrapolation (REx)](https://mlsanity.com/paper/2003.00688/) (2020)
- [This Looks Like That: Deep Learning for Interpretable Image Recognition](https://mlsanity.com/paper/1806.10574/) (2018)
- [Recognition in Terra Incognita](https://mlsanity.com/paper/1807.04975/) (2018)
- [Generalizing to Unseen Domains via Adversarial Data Augmentation](https://mlsanity.com/paper/1805.12018/) (2018)
- [Domain Generalization via Model-Agnostic Learning of Semantic Features](https://mlsanity.com/paper/1910.13580/) (2019)
- [Self-supervised Equivariant Attention Mechanism for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2004.04581/) (2020)
- [Learning to Learn Single Domain Generalization](https://mlsanity.com/paper/2003.13216/) (2020)
- [Right for the Right Reasons: Training Differentiable Models by Constraining their Explanations](https://mlsanity.com/paper/1703.03717/) (2017)
- [Generalizing Across Domains via Cross-Gradient Training](https://mlsanity.com/paper/1804.10745/) (2018)
- [Adversarial Domain Adaptation with Domain Mixup](https://mlsanity.com/paper/1912.01805/) (2019)
- [SMIL: Multimodal Learning with Severely Missing Modality](https://mlsanity.com/paper/2103.05677/) (2021)

## Built on by (most cited first, 14 of 14)

- [Text-Guided Attention is All You Need for Zero-Shot Robustness in Vision-Language Models](https://mlsanity.com/paper/2410.21802/) (2024)
- [DEAL: Disentangle and Localize Concept-level Explanations for VLMs](https://mlsanity.com/paper/2407.14412/) (2024)
- [Gradient-Regularized Out-of-Distribution Detection](https://mlsanity.com/paper/2404.12368/) (2024)
- [Selective Mixup Helps with Distribution Shifts, But Not (Only) because of Mixup](https://mlsanity.com/paper/2305.16817/) (2023)
- [Topology-aware Robust Optimization for Out-of-distribution Generalization](https://mlsanity.com/paper/2307.13943/) (2023)
- [Beyond the Federation: Topology-aware Federated Learning for Generalization to Unseen Clients](https://mlsanity.com/paper/2407.04949/) (2024)
- [Beyond Accuracy: Ensuring Correct Predictions With Correct Rationales](https://mlsanity.com/paper/2411.00132/) (2024)
- [Inside the Visual Mind: Neuroscience-Motivated Concept Circuits for Interpreting and Steering Vision Transformers](https://mlsanity.com/paper/2606.06664/) (2026)
- [Combining Machine Learning Defenses without Conflicts](https://mlsanity.com/paper/2411.09776/) (2024)
- [Adaptive Cascading Network for Continual Test-Time Adaptation](https://mlsanity.com/paper/2407.12240/) (2024)
- [Explanation-Guided Adversarial Training for Robust and Interpretable Models](https://mlsanity.com/paper/2603.01938/) (2026)
- [From Attribution to Action: Jointly ALIGNing Predictions and Explanations](https://mlsanity.com/paper/2511.06944/) (2025)
- [Complementary Text-Guided Attention for Zero-Shot Adversarial Robustness](https://mlsanity.com/paper/2603.18598/) (2026)
- [Consistent Evidence, Robust Recognition: Faithful Attribution Regularization under Geometric Transformations](https://mlsanity.com/paper/2607.23835/) (2026)

## Abstract (arXiv)

As black-box models increasingly power high-stakes applications, a variety of data-driven explanation methods have been introduced. Meanwhile, machine learning models are constantly challenged by distributional shifts. A question naturally arises: Are data-driven explanations robust against out-of-distribution data? Our empirical results show that even though predict correctly, the model might still yield unreliable explanations under distributional shifts. How to develop robust explanations against out-of-distribution data? To address this problem, we propose an end-to-end model-agnostic learning framework Distributionally Robust Explanations (DRE). The key idea is, inspired by self-supervised learning, to fully utilizes the inter-distribution information to provide supervisory signals for the learning of explanations without human annotation. Can robust explanations benefit the model's generalization capability? We conduct extensive experiments on a wide range of tasks and data types, including classification and regression on image and scientific tabular data. Our results demonstrate that the proposed method significantly improves the model's performance in terms of explanation and prediction robustness against distributional shifts.