# Learning Variational Word Masks to Improve the Interpretability of Neural Text Classifiers

arXiv: https://arxiv.org/abs/2010.00667 · HTML: https://mlsanity.com/paper/2010.00667/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Oct 2020.

## Built on by (most cited first, 30 of 35)

- [The Out-of-Distribution Problem in Explainability and Search Methods for Feature Importance Explanations](https://mlsanity.com/paper/2106.00786/) (2021)
- [Towards Improving Adversarial Training of NLP Models](https://mlsanity.com/paper/2109.00544/) (2021)
- [Improving the Faithfulness of Attention-based Explanations with Task-specific Information for Text Classification](https://mlsanity.com/paper/2105.02657/) (2021)
- [Interpreting Deep Learning Models in Natural Language Processing: A Review](https://mlsanity.com/paper/2110.10470/) (2021)
- [Logic Traps in Evaluating Attribution Scores](https://mlsanity.com/paper/2109.05463/) (2021)
- [Towards Explainable Evaluation Metrics for Natural Language Generation](https://mlsanity.com/paper/2203.11131/) (2022)
- [Explaining Neural Network Predictions on Sentence Pairs via Learning Word-Group Masks](https://mlsanity.com/paper/2104.04488/) (2021)
- [Adversarial Training for Improving Model Robustness? Look at Both Prediction and Interpretation](https://mlsanity.com/paper/2203.12709/) (2022)
- [Unsupervised Text Deidentification](https://mlsanity.com/paper/2210.11528/) (2022)
- [Flexible Instance-Specific Rationalization of NLP Models](https://mlsanity.com/paper/2104.08219/) (2021)
- [Normalized AOPC: Fixing Misleading Faithfulness Metrics for Feature Attribution Explainability](https://mlsanity.com/paper/2408.08137/) (2024)
- [Human-Centric Research for NLP: Towards a Definition and Guiding Questions](https://mlsanity.com/paper/2207.04447/) (2022)
- [Towards Faithful Explanations: Boosting Rationalization with Shortcuts Discovery](https://mlsanity.com/paper/2403.07955/) (2024)
- [Local Explanation of Dialogue Response Generation](https://mlsanity.com/paper/2106.06528/) (2021)
- [Human Guided Exploitation of Interpretable Attention Patterns in Summarization and Topic Segmentation](https://mlsanity.com/paper/2112.05364/) (2021)
- [An Information Minimization Based Contrastive Learning Model for Unsupervised Sentence Embeddings Learning](https://mlsanity.com/paper/2209.10951/) (2022)
- [Improving Interpretability via Explicit Word Interaction Graph Layer](https://mlsanity.com/paper/2302.02016/) (2023)
- [JoPA:Explaining Large Language Model's Generation via Joint Prompt Attribution](https://mlsanity.com/paper/2405.20404/) (2024)
- [Hierarchical Interpretation of Neural Text Classification](https://mlsanity.com/paper/2202.09792/) (2022)
- [HOP, UNION, GENERATE: Explainable Multi-hop Reasoning without Rationale Supervision](https://mlsanity.com/paper/2305.14237/) (2023)
- [Exploring the Trade-off Between Model Performance and Explanation Plausibility of Text Classifiers Using Human Rationales](https://mlsanity.com/paper/2404.03098/) (2024)
- [A Multi-Grained Self-Interpretable Symbolic-Neural Model For Single/Multi-Labeled Text Classification](https://mlsanity.com/paper/2303.02860/) (2023)
- [Learning Distribution-Wise Control in Representation Space for Language Models](https://mlsanity.com/paper/2506.06686/) (2025)
- [Explaining the Road Not Taken](https://mlsanity.com/paper/2103.14973/) (2021)
- [Pathologies of Pre-trained Language Models in Few-shot Fine-tuning](https://mlsanity.com/paper/2204.08039/) (2022)
- [Consistent Multi-Granular Rationale Extraction for Explainable Multi-hop Fact Verification](https://mlsanity.com/paper/2305.09400/) (2023)
- [Identifying the Source of Vulnerability in Explanation Discrepancy: A Case Study in Neural Text Classification](https://mlsanity.com/paper/2212.05327/) (2022)
- [Tell Model Where to Attend: Improving Interpretability of Aspect-Based Sentiment Classification via Small Explanation Annotations](https://mlsanity.com/paper/2302.10479/) (2023)
- [SAFR: Neuron Redistribution for Interpretability](https://mlsanity.com/paper/2501.16374/) (2025)
- [Fact in Fragments: Deconstructing Complex Claims via LLM-based Atomic Fact Extraction and Verification](https://mlsanity.com/paper/2506.07446/) (2025)

## Abstract (arXiv)

To build an interpretable neural text classifier, most of the prior work has focused on designing inherently interpretable models or finding faithful explanations. A new line of work on improving model interpretability has just started, and many existing methods require either prior information or human annotations as additional inputs in training. To address this limitation, we propose the variational word mask (VMASK) method to automatically learn task-specific important words and reduce irrelevant information on classification, which ultimately improves the interpretability of model predictions. The proposed method is evaluated with three neural text classifiers (CNN, LSTM, and BERT) on seven benchmark text classification datasets. Experiments show the effectiveness of VMASK in improving both model prediction accuracy and interpretability.