# Explaining Neural Network Predictions on Sentence Pairs via Learning Word-Group Masks

arXiv: https://arxiv.org/abs/2104.04488 · HTML: https://mlsanity.com/paper/2104.04488/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 46 references that are in the map.
- First posted on arXiv 9 Apr 2021.

## Builds on (most cited first, 30 of 35)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [A large annotated corpus for learning natural language inference](https://mlsanity.com/paper/1508.05326/) (2015)
- [Enriching Word Vectors with Subword Information](https://mlsanity.com/paper/1607.04606/) (2016)
- [The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables](https://mlsanity.com/paper/1611.00712/) (2016)
- [A Survey Of Methods For Explaining Black Box Models](https://mlsanity.com/paper/1802.01933/) (2018)
- [Attention is not Explanation](https://mlsanity.com/paper/1902.10186/) (2019)
- [Attention is not not Explanation](https://mlsanity.com/paper/1908.04626/) (2019)
- [e-SNLI: Natural Language Inference with Natural Language Explanations](https://mlsanity.com/paper/1812.01193/) (2018)
- [Evaluating the visualization of what a Deep Neural Network has learned](https://mlsanity.com/paper/1509.06321/) (2015)
- [Is Attention Interpretable?](https://mlsanity.com/paper/1906.03731/) (2019)
- [Consistent Individualized Feature Attribution for Tree Ensembles](https://mlsanity.com/paper/1802.03888/) (2018)
- [Rationalizing Neural Predictions](https://mlsanity.com/paper/1606.04155/) (2016)
- [Learning to Explain: An Information-Theoretic Perspective on Model Interpretation](https://mlsanity.com/paper/1802.07814/) (2018)
- [Understanding Neural Networks through Representation Erasure](https://mlsanity.com/paper/1612.08220/) (2016)
- [A Decomposable Attention Model for Natural Language Inference](https://mlsanity.com/paper/1606.01933/) (2016)
- [Interpretable Neural Predictions with Differentiable Binary Variables](https://mlsanity.com/paper/1905.08160/) (2019)
- [Bilateral Multi-Perspective Matching for Natural Language Sentences](https://mlsanity.com/paper/1702.03814/) (2017)
- [Restricting the Flow: Information Bottlenecks for Attribution](https://mlsanity.com/paper/2001.00396/) (2020)
- [Reasoning about Entailment with Neural Attention](https://mlsanity.com/paper/1509.06664/) (2015)
- [Beyond Word Importance: Contextual Decomposition to Extract Interactions from LSTMs](https://mlsanity.com/paper/1801.05453/) (2018)
- [Hierarchical interpretations for neural network predictions](https://mlsanity.com/paper/1806.05337/) (2018)
- [How do Decisions Emerge across Layers in Neural Models? Interpretation with Differentiable Masking](https://mlsanity.com/paper/2004.14992/) (2020)
- [Generating Hierarchical Explanations on Text Classification via Feature Interaction Detection](https://mlsanity.com/paper/2004.02015/) (2020)
- [Learning Variational Word Masks to Improve the Interpretability of Neural Text Classifiers](https://mlsanity.com/paper/2010.00667/) (2020)
- [Feature Interaction Interpretability: A Case for Explaining Ad-Recommendation Systems via Neural Interaction Detection](https://mlsanity.com/paper/2006.10966/) (2020)

## Built on by (most cited first, 12 of 12)

- [Optimal ablation for interpretability](https://mlsanity.com/paper/2409.09951/) (2024)
- [REV: Information-Theoretic Evaluation of Free-Text Rationales](https://mlsanity.com/paper/2210.04982/) (2022)
- [Towards Explainable Evaluation Metrics for Natural Language Generation](https://mlsanity.com/paper/2203.11131/) (2022)
- [Towards Explainable Evaluation Metrics for Machine Translation](https://mlsanity.com/paper/2306.13041/) (2023)
- [Thermostat: A Large Collection of NLP Model Explanations and Analysis Tools](https://mlsanity.com/paper/2108.13961/) (2021)
- [Local Explanation of Dialogue Response Generation](https://mlsanity.com/paper/2106.06528/) (2021)
- [Evaluating Input Feature Explanations through a Unified Diagnostic Evaluation Framework](https://mlsanity.com/paper/2406.15085/) (2024)
- [Pathologies of Pre-trained Language Models in Few-shot Fine-tuning](https://mlsanity.com/paper/2204.08039/) (2022)
- [Explaining Predictive Uncertainty by Looking Back at Model Explanations](https://mlsanity.com/paper/2201.03742/) (2022)
- [Interpretable Research Replication Prediction via Variational Contextual Consistency Sentence Masking](https://mlsanity.com/paper/2203.14474/) (2022)
- [Explaining Interactions Between Text Spans](https://mlsanity.com/paper/2310.13506/) (2023)
- [Counterfactual Samples Constructing and Training for Commonsense Statements Estimation](https://mlsanity.com/paper/2412.20563/) (2024)

## Abstract (arXiv)

Explaining neural network models is important for increasing their trustworthiness in real-world applications. Most existing methods generate post-hoc explanations for neural network models by identifying individual feature attributions or detecting interactions between adjacent features. However, for models with text pairs as inputs (e.g., paraphrase identification), existing methods are not sufficient to capture feature interactions between two texts and their simple extension of computing all word-pair interactions between two texts is computationally inefficient. In this work, we propose the Group Mask (GMASK) method to implicitly detect word correlations by grouping correlated words from the input text pair together and measure their contribution to the corresponding NLP tasks as a whole. The proposed method is evaluated with two different model architectures (decomposable attention model and BERT) across four datasets, including natural language inference and paraphrase identification tasks. Experiments show the effectiveness of GMASK in providing faithful explanations to these models.