# Embracing data abundance: BookTest Dataset for Reading Comprehension

arXiv: https://arxiv.org/abs/1610.00956 · HTML: https://mlsanity.com/paper/1610.00956/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Oct 2016.

## Built on by (most cited first, 15 of 15)

- [QuALITY: Question Answering with Long Input Texts, Yes!](https://mlsanity.com/paper/2112.08608/) (2021)
- [QA Dataset Explosion: A Taxonomy of NLP Resources for Question Answering and Reading Comprehension](https://mlsanity.com/paper/2107.12708/) (2021)
- [Fantastic Questions and Where to Find Them: FairytaleQA -- An Authentic Dataset for Narrative Comprehension](https://mlsanity.com/paper/2203.13947/) (2022)
- [DFX: A Low-latency Multi-FPGA Appliance for Accelerating Transformer-based Text Generation](https://mlsanity.com/paper/2209.10797/) (2022)
- [Question Answering Survey: Directions, Challenges, Datasets, Evaluation Matrices](https://mlsanity.com/paper/2112.03572/) (2021)
- [TVShowGuess: Character Comprehension in Stories as Speaker Guessing](https://mlsanity.com/paper/2204.07721/) (2022)
- [PQuAD: A Persian Question Answering Dataset](https://mlsanity.com/paper/2202.06219/) (2022)
- [A Survey of Machine Narrative Reading Comprehension Assessments](https://mlsanity.com/paper/2205.00299/) (2022)
- [Are NLP Models Good at Tracing Thoughts: An Overview of Narrative Understanding](https://mlsanity.com/paper/2310.18783/) (2023)
- [More Than Reading Comprehension: A Survey on Datasets and Metrics of Textual Question Answering](https://mlsanity.com/paper/2109.12264/) (2021)
- [Data Augmentation for Biomedical Factoid Question Answering](https://mlsanity.com/paper/2204.04711/) (2022)
- [NaijaRC: A Multi-choice Reading Comprehension Dataset for Nigerian Languages](https://mlsanity.com/paper/2308.09768/) (2023)
- [Native Chinese Reader: A Dataset Towards Native-Level Chinese Machine Reading Comprehension](https://mlsanity.com/paper/2112.06494/) (2021)
- [Recent Advances in Multi-Choice Machine Reading Comprehension: A Survey on Methods and Datasets](https://mlsanity.com/paper/2408.02114/) (2024)
- [Building a Rich Dataset to Empower the Persian Question Answering Systems](https://mlsanity.com/paper/2412.20212/) (2024)

## Abstract (arXiv)

There is a practically unlimited amount of natural language data available. Still, recent work in text comprehension has focused on datasets which are small relative to current computing possibilities. This article is making a case for the community to move to larger data and as a step in that direction it is proposing the BookTest, a new dataset similar to the popular Children's Book Test (CBT), however more than 60 times larger. We show that training on the new data improves the accuracy of our Attention-Sum Reader model on the original CBT test data by a much larger margin than many recent attempts to improve the model architecture. On one version of the dataset our ensemble even exceeds the human baseline provided by Facebook. We then show in our own human study that there is still space for further improvement.