# Contextualized Word Representations for Reading Comprehension

arXiv: https://arxiv.org/abs/1712.03609 · HTML: https://mlsanity.com/paper/1712.03609/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Dec 2017.

## Built on by (most cited first, 5 of 5)

- [Bertinho: Galician BERT Representations](https://mlsanity.com/paper/2103.13799/) (2021)
- [Deep Transfer Learning & Beyond: Transformer Language Models in Information Systems Research](https://mlsanity.com/paper/2110.08975/) (2021)
- [A Frustratingly Easy Improvement for Position Embeddings via Random Padding](https://mlsanity.com/paper/2305.04859/) (2023)
- [Divide & Conquer for Entailment-aware Multi-hop Evidence Retrieval](https://mlsanity.com/paper/2311.02616/) (2023)
- [Perturbation-based Active Learning for Question Answering](https://mlsanity.com/paper/2311.02345/) (2023)

## Abstract (arXiv)

Reading a document and extracting an answer to a question about its content has attracted substantial attention recently. While most work has focused on the interaction between the question and the document, in this work we evaluate the importance of context when the question and document are processed independently. We take a standard neural architecture for this task, and show that by providing rich contextualized word representations from a large pre-trained language model as well as allowing the model to choose between context-dependent and context-independent word representations, we can obtain dramatic improvements and reach performance comparable to state-of-the-art on the competitive SQuAD dataset.