# The Role of Explanatory Value in Natural Language Processing

arXiv: https://arxiv.org/abs/2209.06169 · HTML: https://mlsanity.com/paper/2209.06169/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 75 references that are in the map.
- First posted on arXiv 13 Sep 2022.

## Builds on (most cited first, 17 of 17)

- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [Modeling Context in Referring Expressions](https://mlsanity.com/paper/1608.00272/) (2016)
- [Beyond Accuracy: Behavioral Testing of NLP models with CheckList](https://mlsanity.com/paper/2005.04118/) (2020)
- [nocaps: novel object captioning at scale](https://mlsanity.com/paper/1812.08658/) (2018)
- [Causal Abstractions of Neural Networks](https://mlsanity.com/paper/2106.02997/) (2021)
- [Rationalizing Neural Predictions](https://mlsanity.com/paper/1606.04155/) (2016)
- [Evaluation of Text Generation: A Survey](https://mlsanity.com/paper/2006.14799/) (2020)
- [Information-Theoretic Probing with Minimum Description Length](https://mlsanity.com/paper/2003.12298/) (2020)
- [A Survey of Evaluation Metrics Used for NLG Systems](https://mlsanity.com/paper/2008.12009/) (2020)
- [Comprehension-guided referring expressions](https://mlsanity.com/paper/1701.03439/) (2017)
- [Cognitive Psychology for Deep Neural Networks: A Shape Bias Case Study](https://mlsanity.com/paper/1706.08606/) (2017)
- [When redundancy is useful: A Bayesian approach to 'overinformative' referring expressions](https://mlsanity.com/paper/1903.08237/) (2019)
- [Non-neural Models Matter: A Re-evaluation of Neural Referring Expression Generation Systems](https://mlsanity.com/paper/2203.08274/) (2022)
- [Referring Expression Generation Using Entity Profiles](https://mlsanity.com/paper/1909.01528/) (2019)
- [What can Neural Referential Form Selectors Learn?](https://mlsanity.com/paper/2108.06806/) (2021)

## Abstract (arXiv)

A key aim of science is explanation, yet the idea of explaining language phenomena has taken a backseat in mainstream Natural Language Processing (NLP) and many other areas of Artificial Intelligence. I argue that explanation of linguistic behaviour should be a main goal of NLP, and that this is not the same as making NLP models explainable. To illustrate these ideas, some recent models of human language production are compared with each other. I conclude by asking what it would mean for NLP research and institutional policies if our community took explanatory value seriously, while heeding some possible pitfalls.