# ArchivalQA: A Large-scale Benchmark Dataset for Open Domain Question Answering over Historical News Collections

arXiv: https://arxiv.org/abs/2109.03438 · HTML: https://mlsanity.com/paper/2109.03438/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Builds on 41 of its 79 references that are in the map.
- First posted on arXiv 8 Sep 2021.

## Builds on (most cited first, 30 of 41)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2004.04906/) (2020)
- [TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension](https://mlsanity.com/paper/1705.03551/) (2017)
- [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://mlsanity.com/paper/1909.11942/) (2019)
- [MS MARCO: A Human Generated MAchine Reading COmprehension Dataset](https://mlsanity.com/paper/1611.09268/) (2016)
- [Know What You Don't Know: Unanswerable Questions for SQuAD](https://mlsanity.com/paper/1806.03822/) (2018)
- [PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization](https://mlsanity.com/paper/1912.08777/) (2019)
- [Teaching Machines to Read and Comprehend](https://mlsanity.com/paper/1506.03340/) (2015)
- [Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyond](https://mlsanity.com/paper/1602.06023/) (2016)
- [Reading Wikipedia to Answer Open-Domain Questions](https://mlsanity.com/paper/1704.00051/) (2017)
- [Latent Retrieval for Weakly Supervised Open Domain Question Answering](https://mlsanity.com/paper/1906.00300/) (2019)
- [CoQA: A Conversational Question Answering Challenge](https://mlsanity.com/paper/1808.07042/) (2018)
- [The NarrativeQA Reading Comprehension Challenge](https://mlsanity.com/paper/1712.07040/) (2017)
- [RocketQA: An Optimized Training Approach to Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2010.08191/) (2020)
- [AmbigQA: Answering Ambiguous Open-domain Questions](https://mlsanity.com/paper/2004.10645/) (2020)
- [NewsQA: A Machine Comprehension Dataset](https://mlsanity.com/paper/1611.09830/) (2016)
- [SearchQA: A New Q&A Dataset Augmented with Context from a Search Engine](https://mlsanity.com/paper/1704.05179/) (2017)
- [ReCoRD: Bridging the Gap between Human and Machine Commonsense Reading Comprehension](https://mlsanity.com/paper/1810.12885/) (2018)
- [Retrieving and Reading: A Comprehensive Survey on Open-domain Question Answering](https://mlsanity.com/paper/2101.00774/) (2021)
- [End-to-End Open-Domain Question Answering with BERTserini](https://mlsanity.com/paper/1902.01718/) (2019)
- [Exploring Models and Data for Image Question Answering](https://mlsanity.com/paper/1505.02074/) (2015)
- [Question Answering Over Temporal Knowledge Graphs](https://mlsanity.com/paper/2106.01515/) (2021)
- [Training Question Answering Models From Synthetic Data](https://mlsanity.com/paper/2002.09599/) (2020)
- [QA Dataset Explosion: A Taxonomy of NLP Resources for Question Answering and Reading Comprehension](https://mlsanity.com/paper/2107.12708/) (2021)
- [End-to-End Synthetic Data Generation for Domain Adaptation of Question Answering Systems](https://mlsanity.com/paper/2010.06028/) (2020)
- [Deep Reinforcement Learning for Mention-Ranking Coreference Models](https://mlsanity.com/paper/1609.08667/) (2016)
- [Unsupervised Question Answering by Cloze Translation](https://mlsanity.com/paper/1906.04980/) (2019)

## Built on by (most cited first, 30 of 30)

- [Benchmark Data Contamination of Large Language Models: A Survey](https://mlsanity.com/paper/2406.04244/) (2024)
- [SPIQA: A Dataset for Multimodal Question Answering on Scientific Papers](https://mlsanity.com/paper/2407.09413/) (2024)
- [Meta-Learning Online Adaptation of Language Models](https://mlsanity.com/paper/2305.15076/) (2023)
- [Online Adaptation of Language Models with a Memory of Amortized Contexts](https://mlsanity.com/paper/2403.04317/) (2024)
- [WikiBigEdit: Understanding the Limits of Lifelong Knowledge Editing in LLMs](https://mlsanity.com/paper/2503.05683/) (2025)
- [ArabicaQA: A Comprehensive Dataset for Arabic Question Answering](https://mlsanity.com/paper/2403.17848/) (2024)
- [TempRetriever: Fusion-based Temporal Dense Passage Retrieval for Time-Sensitive Questions](https://mlsanity.com/paper/2502.21024/) (2025)
- [It's High Time: A Survey of Temporal Question Answering](https://mlsanity.com/paper/2505.20243/) (2025)
- [Rankify: A Comprehensive Python Toolkit for Retrieval, Re-Ranking, and Retrieval-Augmented Generation](https://mlsanity.com/paper/2502.02464/) (2025)
- [Once Upon a textitTime in textitGraph: Relative-Time Pretraining for Complex Temporal Reasoning](https://mlsanity.com/paper/2310.14709/) (2023)
- [A Study into Investigating Temporal Robustness of LLMs](https://mlsanity.com/paper/2503.17073/) (2025)
- [ChroniclingAmericaQA: A Large-scale Question Answering Dataset based on Historical American Newspaper Pages](https://mlsanity.com/paper/2403.17859/) (2024)
- ["I Never Said That": A dataset, taxonomy and baselines on response clarity classification](https://mlsanity.com/paper/2409.13879/) (2024)
- [ASRank: Zero-Shot Re-Ranking with Answer Scent for Document Retrieval](https://mlsanity.com/paper/2501.15245/) (2025)
- [Unlocking Temporal Question Answering for Large Language Models with Tailor-Made Reasoning Logic](https://mlsanity.com/paper/2305.15014/) (2023)
- [Temporal Blind Spots in Large Language Models](https://mlsanity.com/paper/2401.12078/) (2024)
- [TriviaHG: A Dataset for Automatic Hint Generation from Factoid Questions](https://mlsanity.com/paper/2403.18426/) (2024)
- [RoQLlama: A Lightweight Romanian Adapted Language Model](https://mlsanity.com/paper/2410.04269/) (2024)
- [CMT: A Memory Compression Method for Continual Knowledge Learning of Large Language Models](https://mlsanity.com/paper/2412.07393/) (2024)
- [Memory Bank Compression for Continual Adaptation of Large Language Models](https://mlsanity.com/paper/2601.00756/) (2026)
- [Taxonomy of the Retrieval System Framework: Pitfalls and Paradigms](https://mlsanity.com/paper/2601.20131/) (2026)
- [Continual Learning for Temporal-Sensitive Question Answering](https://mlsanity.com/paper/2407.12470/) (2024)
- [Latent Paraphrasing: Perturbation on Layers Improves Knowledge Injection in Language Models](https://mlsanity.com/paper/2411.00686/) (2024)
- [Evaluating Answer Reranking Strategies in Time-sensitive Question Answering](https://mlsanity.com/paper/2503.04972/) (2025)
- [Does the Generator Mind its Contexts? An Analysis of Generative Model Faithfulness under Context Transfer](https://mlsanity.com/paper/2402.14488/) (2024)
- [Detecting Temporal Ambiguity in Questions](https://mlsanity.com/paper/2409.17046/) (2024)
- [Few-Shot Joint Multimodal Entity-Relation Extraction via Knowledge-Enhanced Cross-modal Prompt Model](https://mlsanity.com/paper/2410.14225/) (2024)
- [Evaluating List Construction and Temporal Understanding capabilities of Large Language Models](https://mlsanity.com/paper/2506.21783/) (2025)
- [Evaluating Temporal Consistency in Multi-Turn Language Models](https://mlsanity.com/paper/2604.23051/) (2026)
- [How often do Answers Change? Estimating Recency Requirements in Question Answering](https://mlsanity.com/paper/2603.16544/) (2026)

## Abstract (arXiv)

In the last few years, open-domain question answering (ODQA) has advanced rapidly due to the development of deep learning techniques and the availability of large-scale QA datasets. However, the current datasets are essentially designed for synchronic document collections (e.g., Wikipedia). Temporal news collections such as long-term news archives spanning several decades, are rarely used in training the models despite they are quite valuable for our society. To foster the research in the field of ODQA on such historical collections, we present ArchivalQA, a large question answering dataset consisting of 532,444 question-answer pairs which is designed for temporal news QA. We divide our dataset into four subparts based on the question difficulty levels and the containment of temporal expressions, which we believe are useful for training and testing ODQA systems characterized by different strengths and abilities. The novel QA dataset-constructing framework that we introduce can be also applied to generate non-ambiguous questions of good quality over other types of temporal document collections.