# Multilingual Multi-Aspect Explainability Analyses on Machine Reading Comprehension Models

arXiv: https://arxiv.org/abs/2108.11574 · HTML: https://mlsanity.com/paper/2108.11574/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 33 references that are in the map.
- First posted on arXiv 26 Aug 2021.

## Builds on (most cited first, 22 of 22)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering](https://mlsanity.com/paper/1809.09600/) (2018)
- [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://mlsanity.com/paper/1909.11942/) (2019)
- [Know What You Don't Know: Unanswerable Questions for SQuAD](https://mlsanity.com/paper/1806.03822/) (2018)
- [Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI](https://mlsanity.com/paper/1910.10045/) (2019)
- [Attention is not Explanation](https://mlsanity.com/paper/1902.10186/) (2019)
- [Attention is not not Explanation](https://mlsanity.com/paper/1908.04626/) (2019)
- [Captum: A unified and generic model interpretability library for PyTorch](https://mlsanity.com/paper/2009.07896/) (2020)
- [Interpretable machine learning: definitions, methods, and applications](https://mlsanity.com/paper/1901.04592/) (2019)
- [Revealing the Dark Secrets of BERT](https://mlsanity.com/paper/1908.08593/) (2019)
- [A Multiscale Visualization of Attention in the Transformer Model](https://mlsanity.com/paper/1906.05714/) (2019)
- [Probing Pretrained Language Models for Lexical Semantics](https://mlsanity.com/paper/2010.05731/) (2020)
- [The elephant in the interpretability room: Why use attention as explanation when we have saliency methods?](https://mlsanity.com/paper/2010.05607/) (2020)
- [A Span-Extraction Dataset for Chinese Machine Reading Comprehension](https://mlsanity.com/paper/1810.07366/) (2018)
- [Gated-Attention Readers for Text Comprehension](https://mlsanity.com/paper/1606.01549/) (2016)
- [Attention-over-Attention Neural Networks for Reading Comprehension](https://mlsanity.com/paper/1607.04423/) (2016)
- [Text Understanding with the Attention Sum Reader Network](https://mlsanity.com/paper/1603.01547/) (2016)
- [ExpMRC: Explainability Evaluation for Machine Reading Comprehension](https://mlsanity.com/paper/2105.04126/) (2021)

## Built on by (most cited first, 5 of 5)

- [TextPruner: A Model Pruning Toolkit for Pre-Trained Language Models](https://mlsanity.com/paper/2203.15996/) (2022)
- [Evaluating Large Language Models on Multimodal Chemistry Olympiad Exams](https://mlsanity.com/paper/2512.14989/) (2025)
- [An Interpretability Evaluation Benchmark for Pre-trained Language Models](https://mlsanity.com/paper/2207.13948/) (2022)
- [Parallel Universes, Parallel Languages: A Comprehensive Study on LLM-based Multilingual Counterfactual Example Generation](https://mlsanity.com/paper/2601.00263/) (2026)
- [Visualizing attention zones in machine reading comprehension models](https://mlsanity.com/paper/2410.20652/) (2024)

## Abstract (arXiv)

Achieving human-level performance on some of the Machine Reading Comprehension (MRC) datasets is no longer challenging with the help of powerful Pre-trained Language Models (PLMs). However, the internal mechanism of these artifacts remains unclear, placing an obstacle for further understanding these models. This paper focuses on conducting a series of analytical experiments to examine the relations between the multi-head self-attention and the final MRC system performance, revealing the potential explainability in PLM-based MRC models. To ensure the robustness of the analyses, we perform our experiments in a multilingual way on top of various PLMs. We discover that passage-to-question and passage understanding attentions are the most important ones in the question answering process, showing strong correlations to the final performance than other parts. Through comprehensive visualizations and case studies, we also observe several general findings on the attention maps, which can be helpful to understand how these models solve the questions.