# Towards Faithful Explanations: Boosting Rationalization with Shortcuts Discovery

arXiv: https://arxiv.org/abs/2403.07955 · HTML: https://mlsanity.com/paper/2403.07955/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 44 references that are in the map.
- First posted on arXiv 12 Mar 2024.

## Builds on (most cited first, 30 of 31)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Billion-scale similarity search with GPUs](https://mlsanity.com/paper/1702.08734/) (2017)
- [BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions](https://mlsanity.com/paper/1905.10044/) (2019)
- [FEVER: a large-scale dataset for Fact Extraction and VERification](https://mlsanity.com/paper/1803.05355/) (2018)
- [Deep Variational Information Bottleneck](https://mlsanity.com/paper/1612.00410/) (2016)
- [ERASER: A Benchmark to Evaluate Rationalized NLP Models](https://mlsanity.com/paper/1911.03429/) (2019)
- [Pretrained Transformers Improve Out-of-Distribution Robustness](https://mlsanity.com/paper/2004.06100/) (2020)
- [Rationalizing Neural Predictions](https://mlsanity.com/paper/1606.04155/) (2016)
- [Discovering Invariant Rationales for Graph Neural Networks](https://mlsanity.com/paper/2201.12872/) (2022)
- [Invariant Rationalization](https://mlsanity.com/paper/2003.09772/) (2020)
- [Interpretable Neural Predictions with Differentiable Binary Variables](https://mlsanity.com/paper/1905.08160/) (2019)
- [Rethinking Cooperative Rationalization: Introspective Extraction and Complement Control](https://mlsanity.com/paper/1910.13294/) (2019)
- [An Information Bottleneck Approach for Controlling Conciseness in Rationale Extraction](https://mlsanity.com/paper/2005.00652/) (2020)
- [Inferring Which Medical Treatments Work from Reports of Clinical Trials](https://mlsanity.com/paper/1904.01606/) (2019)
- [Deriving Machine Attention from Human Rationales](https://mlsanity.com/paper/1808.09367/) (2018)
- [Understanding Interlocking Dynamics of Cooperative Rationalization](https://mlsanity.com/paper/2110.13880/) (2021)
- [Learning Variational Word Masks to Improve the Interpretability of Neural Text Classifiers](https://mlsanity.com/paper/2010.00667/) (2020)
- [A Game Theoretic Approach to Class-wise Selective Rationalization](https://mlsanity.com/paper/1910.12853/) (2019)
- [UNIREX: A Unified Learning Framework for Language Model Rationale Extraction](https://mlsanity.com/paper/2112.08802/) (2021)
- [Learning from the Best: Rationalizing Prediction by Adversarial Information Calibration](https://mlsanity.com/paper/2012.08884/) (2020)
- [What to Learn, and How: Toward Effective Learning from Rationales](https://mlsanity.com/paper/2112.00071/) (2021)
- [FiD-Ex: Improving Sequence-to-Sequence Models for Extractive Rationale Generation](https://mlsanity.com/paper/2012.15482/) (2020)
- [D-Separation for Causal Self-Explanation](https://mlsanity.com/paper/2309.13391/) (2023)
- [Rationalization through Concepts](https://mlsanity.com/paper/2105.04837/) (2021)
- [Weakly- and Semi-supervised Evidence Extraction](https://mlsanity.com/paper/2011.01459/) (2020)
- [Distribution Matching for Rationalization](https://mlsanity.com/paper/2106.00320/) (2021)
- [Self-training with Few-shot Rationalization: Teacher Explanations Aid Student in Few-shot NLU](https://mlsanity.com/paper/2109.08259/) (2021)
- [Unifying Model Explainability and Robustness for Joint Text Classification and Rationale Extraction](https://mlsanity.com/paper/2112.10424/) (2021)

## Built on by (most cited first, 8 of 8)

- [Why and How LLMs Hallucinate: Connecting the Dots with Subsequence Associations](https://mlsanity.com/paper/2504.12691/) (2025)
- [Is the MMI Criterion Necessary for Interpretability? Degenerating Non-causal Features to Plain Noise for Self-Rationalization](https://mlsanity.com/paper/2410.06003/) (2024)
- [Physics-informed Temporal Alignment for Auto-regressive PDE Foundation Models](https://mlsanity.com/paper/2505.10930/) (2025)
- [GNN Explanations that do not Explain and How to find Them](https://mlsanity.com/paper/2601.20815/) (2026)
- [Boosting Explainability through Selective Rationalization in Pre-trained Language Models](https://mlsanity.com/paper/2501.03182/) (2025)
- [Learnable Game-theoretic Policy Optimization for Data-centric Self-explanation Rationalization](https://mlsanity.com/paper/2510.13393/) (2025)
- [The Pragmatic Frames of Spurious Correlations in Machine Learning: Interpreting How and Why They Matter](https://mlsanity.com/paper/2411.04696/) (2024)
- [Learn from A Rationalist: Distilling Intermediate Interpretable Rationales](https://mlsanity.com/paper/2601.22531/) (2026)

## Abstract (arXiv)

The remarkable success in neural networks provokes the selective rationalization. It explains the prediction results by identifying a small subset of the inputs sufficient to support them. Since existing methods still suffer from adopting the shortcuts in data to compose rationales and limited large-scale annotated rationales by human, in this paper, we propose a Shortcuts-fused Selective Rationalization (SSR) method, which boosts the rationalization by discovering and exploiting potential shortcuts. Specifically, SSR first designs a shortcuts discovery approach to detect several potential shortcuts. Then, by introducing the identified shortcuts, we propose two strategies to mitigate the problem of utilizing shortcuts to compose rationales. Finally, we develop two data augmentations methods to close the gap in the number of annotated rationales. Extensive experimental results on real-world datasets clearly validate the effectiveness of our proposed method.