# Quasar: Datasets for Question Answering by Search and Reading

arXiv: https://arxiv.org/abs/1707.03904 · HTML: https://mlsanity.com/paper/1707.03904/

## Key facts

- Cited by 50 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Jul 2017.

## Built on by (most cited first, 30 of 50)

- [DiffuSeq: Sequence to Sequence Text Generation with Diffusion Models](https://mlsanity.com/paper/2210.08933/) (2022)
- [DiffusionBERT: Improving Generative Masked Language Models with Diffusion Models](https://mlsanity.com/paper/2211.15029/) (2022)
- [A Reparameterized Discrete Diffusion Model for Text Generation](https://mlsanity.com/paper/2302.05737/) (2023)
- [WebSRC: A Dataset for Web-Based Structural Reading Comprehension](https://mlsanity.com/paper/2101.09465/) (2021)
- [TESS: Text-to-Text Self-Conditioned Simplex Diffusion](https://mlsanity.com/paper/2305.08379/) (2023)
- [SeqDiffuSeq: Text Diffusion with Encoder-Decoder Transformers](https://mlsanity.com/paper/2212.10325/) (2022)
- [Empowering Diffusion Models on the Embedding Space for Text Generation](https://mlsanity.com/paper/2212.09412/) (2022)
- [ArchivalQA: A Large-scale Benchmark Dataset for Open Domain Question Answering over Historical News Collections](https://mlsanity.com/paper/2109.03438/) (2021)
- [English Machine Reading Comprehension Datasets: A Survey](https://mlsanity.com/paper/2101.10421/) (2021)
- [Modern Question Answering Datasets and Benchmarks: A Survey](https://mlsanity.com/paper/2206.15030/) (2022)
- [Database Reasoning Over Text](https://mlsanity.com/paper/2106.01074/) (2021)
- [DLM-One: Diffusion Language Models for One-Step Sequence Generation](https://mlsanity.com/paper/2506.00290/) (2025)
- [Rethinking Search: Making Domain Experts out of Dilettantes](https://mlsanity.com/paper/2105.02274/) (2021)
- [Question Answering Survey: Directions, Challenges, Datasets, Evaluation Matrices](https://mlsanity.com/paper/2112.03572/) (2021)
- [Narrative Question Answering with Cutting-Edge Open-Domain QA Techniques: A Comprehensive Study](https://mlsanity.com/paper/2106.03826/) (2021)
- [SportQA: A Benchmark for Sports Understanding in Large Language Models](https://mlsanity.com/paper/2402.15862/) (2024)
- [Augmenting Multi-Agent Communication with State Delta Trajectory](https://mlsanity.com/paper/2506.19209/) (2025)
- [Answering Open-Domain Multi-Answer Questions via a Recall-then-Verify Framework](https://mlsanity.com/paper/2110.08544/) (2021)
- [EduQG: A Multi-format Multiple Choice Dataset for the Educational Domain](https://mlsanity.com/paper/2210.06104/) (2022)
- [Can Diffusion Model Achieve Better Performance in Text Generation? Bridging the Gap between Training and Inference!](https://mlsanity.com/paper/2305.04465/) (2023)
- [More Than Reading Comprehension: A Survey on Datasets and Metrics of Textual Question Answering](https://mlsanity.com/paper/2109.12264/) (2021)
- [Smoothie: Smoothing Diffusion on Token Embeddings for Text Generation](https://mlsanity.com/paper/2505.18853/) (2025)
- [Interactive Question Answering Systems: Literature Review](https://mlsanity.com/paper/2209.01621/) (2022)
- [Sports Intelligence: Assessing the Sports Understanding Capabilities of Language Models through Question Answering from Text to Video](https://mlsanity.com/paper/2406.14877/) (2024)
- [Cross-language Sentence Selection via Data Augmentation and Rationale Training](https://mlsanity.com/paper/2106.02293/) (2021)
- [To Adapt or to Annotate: Challenges and Interventions for Domain Adaptation in Open-Domain Question Answering](https://mlsanity.com/paper/2212.10381/) (2022)
- [InfoDiffusion: Information Entropy Aware Diffusion Process for Non-Autoregressive Text Generation](https://mlsanity.com/paper/2310.11976/) (2023)
- [Conditional [MASK] Discrete Diffusion Language Model](https://mlsanity.com/paper/2411.06438/) (2024)
- [ComQA:Compositional Question Answering via Hierarchical Graph Neural Networks](https://mlsanity.com/paper/2101.06400/) (2021)
- [Weakly-Supervised Open-Retrieval Conversational Question Answering](https://mlsanity.com/paper/2103.02537/) (2021)

## Abstract (arXiv)

We present two new large-scale datasets aimed at evaluating systems designed to comprehend a natural language query and extract its answer from a large corpus of text. The Quasar-S dataset consists of 37000 cloze-style (fill-in-the-gap) queries constructed from definitions of software entity tags on the popular website Stack Overflow. The posts and comments on the website serve as the background corpus for answering the cloze questions. The Quasar-T dataset consists of 43000 open-domain trivia questions and their answers obtained from various internet sources. ClueWeb09 serves as the background corpus for extracting these answers. We pose these datasets as a challenge for two related subtasks of factoid Question Answering: (1) searching for relevant pieces of text that include the correct answer to a query, and (2) reading the retrieved text to answer the query. We also describe a retrieval system for extracting relevant sentences and documents from the corpus given a query, and include these in the release for researchers wishing to only focus on (2). We evaluate several baselines on both datasets, ranging from simple heuristics to powerful neural models, and show that these lag behind human performance by 16.4% and 32.1% for Quasar-S and -T respectively. The datasets are available at https://github.com/bdhingra/quasar .