# Learning from the Best: Rationalizing Prediction by Adversarial Information Calibration

arXiv: https://arxiv.org/abs/2012.08884 · HTML: https://mlsanity.com/paper/2012.08884/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Dec 2020.

## Built on by (most cited first, 25 of 25)

- [A Survey on Uncertainty Quantification of Large Language Models: Taxonomy, Open Research Challenges, and Future Directions](https://mlsanity.com/paper/2412.05563/) (2024)
- [Explaining black box text modules in natural language with language models](https://mlsanity.com/paper/2305.09863/) (2023)
- [Augmenting Interpretable Models with LLMs during Training](https://mlsanity.com/paper/2209.11799/) (2022)
- [Understanding Interlocking Dynamics of Cooperative Rationalization](https://mlsanity.com/paper/2110.13880/) (2021)
- [Can Rationalization Improve Robustness?](https://mlsanity.com/paper/2204.11790/) (2022)
- [Local Interpretations for Explainable Natural Language Processing: A Survey](https://mlsanity.com/paper/2103.11072/) (2021)
- [What to Learn, and How: Toward Effective Learning from Rationales](https://mlsanity.com/paper/2112.00071/) (2021)
- [Knowledge-Grounded Self-Rationalization via Extractive and Natural Language Explanations](https://mlsanity.com/paper/2106.13876/) (2021)
- [Towards Trustworthy Explanation: On Causal Rationalization](https://mlsanity.com/paper/2306.14115/) (2023)
- [FR: Folded Rationalization with a Unified Encoder](https://mlsanity.com/paper/2209.08285/) (2022)
- [Exploring Faithful Rationale for Multi-hop Fact Verification via Salience-Aware Graph Learning](https://mlsanity.com/paper/2212.01060/) (2022)
- [Unifying Model Explainability and Robustness for Joint Text Classification and Rationale Extraction](https://mlsanity.com/paper/2112.10424/) (2021)
- [Is the MMI Criterion Necessary for Interpretability? Degenerating Non-causal Features to Plain Noise for Self-Rationalization](https://mlsanity.com/paper/2410.06003/) (2024)
- [Towards Faithful Explanations: Boosting Rationalization with Shortcuts Discovery](https://mlsanity.com/paper/2403.07955/) (2024)
- [Cooperative Classification and Rationalization for Graph Generalization](https://mlsanity.com/paper/2403.06239/) (2024)
- [Controlling Text Edition by Changing Answers of Specific Questions](https://mlsanity.com/paper/2105.11018/) (2021)
- [A Survey of Computer Vision Technologies In Urban and Controlled-environment Agriculture](https://mlsanity.com/paper/2210.11318/) (2022)
- [Breaking Free from MMI: A New Frontier in Rationalization by Probing Input Utilization](https://mlsanity.com/paper/2503.06202/) (2025)
- [Adversarial Cooperative Rationalization: The Risk of Spurious Correlations in Even Clean Datasets](https://mlsanity.com/paper/2505.02118/) (2025)
- [Rationalizing Predictions by Adversarial Information Calibration](https://mlsanity.com/paper/2301.06009/) (2023)
- [Rationale production to support clinical decision-making](https://mlsanity.com/paper/2111.07611/) (2021)
- [SPARSEFIT: Few-shot Prompting with Sparse Fine-tuning for Jointly Generating Predictions and Natural Language Explanations](https://mlsanity.com/paper/2305.13235/) (2023)
- [Text Rationalization for Robust Causal Effect Estimation](https://mlsanity.com/paper/2512.05373/) (2025)
- [PEACH: Pretrained-embedding Explanation Across Contextual and Hierarchical Structure](https://mlsanity.com/paper/2404.13645/) (2024)
- [Explaining Patterns in Data with Language Models via Interpretable Autoprompting](https://mlsanity.com/paper/2210.01848/) (2022)

## Abstract (arXiv)

Explaining the predictions of AI models is paramount in safety-critical applications, such as in legal or medical domains. One form of explanation for a prediction is an extractive rationale, i.e., a subset of features of an instance that lead the model to give its prediction on the instance. Previous works on generating extractive rationales usually employ a two-phase model: a selector that selects the most important features (i.e., the rationale) followed by a predictor that makes the prediction based exclusively on the selected features. One disadvantage of these works is that the main signal for learning to select features comes from the comparison of the answers given by the predictor and the ground-truth answers. In this work, we propose to squeeze more information from the predictor via an information calibration method. More precisely, we train two models jointly: one is a typical neural model that solves the task at hand in an accurate but black-box manner, and the other is a selector-predictor model that additionally produces a rationale for its prediction. The first model is used as a guide to the second model. We use an adversarial-based technique to calibrate the information extracted by the two models such that the difference between them is an indicator of the missed or over-selected features. In addition, for natural language tasks, we propose to use a language-model-based regularizer to encourage the extraction of fluent rationales. Experimental results on a sentiment analysis task as well as on three tasks from the legal domain show the effectiveness of our approach to rationale extraction.