# Deriving Machine Attention from Human Rationales

arXiv: https://arxiv.org/abs/1808.09367 · HTML: https://mlsanity.com/paper/1808.09367/

## Key facts

- Cited by 54 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Aug 2018.

## Built on by (most cited first, 30 of 54)

- [Teach Me to Explain: A Review of Datasets for Explainable Natural Language Processing](https://mlsanity.com/paper/2102.12060/) (2021)
- [When Can Models Learn From Explanations? A Formal Framework for Understanding the Roles of Explanation Data](https://mlsanity.com/paper/2102.02201/) (2021)
- [On the Explainability of Natural Language Processing Deep Models](https://mlsanity.com/paper/2210.06929/) (2022)
- [Understanding Interlocking Dynamics of Cooperative Rationalization](https://mlsanity.com/paper/2110.13880/) (2021)
- [Going Beyond XAI: A Systematic Survey for Explanation-Guided Learning](https://mlsanity.com/paper/2212.03954/) (2022)
- [Towards Robust Classification Model by Counterfactual and Invariant Data Generation](https://mlsanity.com/paper/2106.01127/) (2021)
- [Do Context-Aware Translation Models Pay the Right Attention?](https://mlsanity.com/paper/2105.06977/) (2021)
- [D-Separation for Causal Self-Explanation](https://mlsanity.com/paper/2309.13391/) (2023)
- [Rationalization through Concepts](https://mlsanity.com/paper/2105.04837/) (2021)
- [SPECTRA: Sparse Structured Text Rationalization](https://mlsanity.com/paper/2109.04552/) (2021)
- [Towards Trustworthy Explanation: On Causal Rationalization](https://mlsanity.com/paper/2306.14115/) (2023)
- [FR: Folded Rationalization with a Unified Encoder](https://mlsanity.com/paper/2209.08285/) (2022)
- [Predict then Interpolate: A Simple Algorithm to Learn Stable Classifiers](https://mlsanity.com/paper/2105.12628/) (2021)
- [A survey on improving NLP models with human explanations](https://mlsanity.com/paper/2204.08892/) (2022)
- [Enhancing the Rationale-Input Alignment for Self-explaining Rationalization](https://mlsanity.com/paper/2312.04103/) (2023)
- [Distribution Matching for Rationalization](https://mlsanity.com/paper/2106.00320/) (2021)
- [Decoupled Rationalization with Asymmetric Learning Rates: A Flexible Lipschitz Restraint](https://mlsanity.com/paper/2305.13599/) (2023)
- [MGR: Multi-generator Based Rationalization](https://mlsanity.com/paper/2305.04492/) (2023)
- [CREW: Facilitating Human-AI Teaming Research](https://mlsanity.com/paper/2408.00170/) (2024)
- [Making a (Counterfactual) Difference One Rationale at a Time](https://mlsanity.com/paper/2201.05177/) (2022)
- [Navigating the Shortcut Maze: A Comprehensive Analysis of Shortcut Learning in Text Classification by Language Models](https://mlsanity.com/paper/2409.17455/) (2024)
- [Exploring Distantly-Labeled Rationales in Neural Network Models](https://mlsanity.com/paper/2106.01809/) (2021)
- [Towards Faithful Explanations: Boosting Rationalization with Shortcuts Discovery](https://mlsanity.com/paper/2403.07955/) (2024)
- [Few-Shot Semantic Parsing for New Predicates](https://mlsanity.com/paper/2101.10708/) (2021)
- [Why Is It Hate Speech? Masked Rationale Prediction for Explainable Hate Speech Detection](https://mlsanity.com/paper/2211.00243/) (2022)
- [A Survey on Human-AI Collaboration with Large Foundation Models](https://mlsanity.com/paper/2403.04931/) (2024)
- [Nutri-bullets: Summarizing Health Studies by Composing Segments](https://mlsanity.com/paper/2103.11921/) (2021)
- [Plausible Extractive Rationalization through Semi-Supervised Entailment Signal](https://mlsanity.com/paper/2402.08479/) (2024)
- [Breaking Free from MMI: A New Frontier in Rationalization by Probing Input Utilization](https://mlsanity.com/paper/2503.06202/) (2025)
- [Adversarial Cooperative Rationalization: The Risk of Spurious Correlations in Even Clean Datasets](https://mlsanity.com/paper/2505.02118/) (2025)

## Abstract (arXiv)

Attention-based models are successful when trained on large amounts of data. In this paper, we demonstrate that even in the low-resource scenario, attention can be learned effectively. To this end, we start with discrete human-annotated rationales and map them into continuous attention. Our central hypothesis is that this mapping is general across domains, and thus can be transferred from resource-rich domains to low-resource ones. Our model jointly learns a domain-invariant representation and induces the desired mapping between rationales and attention. Our empirical results validate this hypothesis and show that our approach delivers significant gains over state-of-the-art baselines, yielding over 15% average error reduction on benchmark datasets.