mlsanityData to 7 Oct 2026

PAQ: 65 Million Probably-Asked Questions and What You Can Do With Them

arXiv 2102.07033 ↗First posted 13 Feb 2021cs.CL cs.AI cs.LGcited by 198

Not on the map. Why?

Abstract (arXiv)

Open-domain Question Answering models which directly leverage question-answer (QA) pairs, such as closed-book QA (CBQA) models and QA-pair retrievers, show promise in terms of speed and memory compared to conventional models which retrieve and read from text corpora. QA-pair retrievers also offer interpretable answers, a high degree of control, and are trivial to update at test time with new knowledge. However, these models lack the accuracy of retrieve-and-read systems, as substantially less knowledge is covered by the available QA-pairs relative to text corpora like Wikipedia. To facilitate improved QA-pair models, we introduce Probably Asked Questions (PAQ), a very large resource of 65M automatically-generated QA-pairs. We introduce a new QA-pair retriever, RePAQ, to complement PAQ. We find that PAQ preempts and caches test questions, enabling RePAQ to match the accuracy of recent retrieve-and-read models, whilst being significantly faster. Using PAQ, we train CBQA models which outperform comparable baselines by 5%, but trail RePAQ by over 15%, indicating the effectiveness of explicit retrieval. RePAQ can be configured for size (under 500MB) or speed (over 1K questions per second) whilst retaining high accuracy. Lastly, we demonstrate RePAQ's strength at selective QA, abstaining from answering when it is likely to be incorrect. This enables RePAQ to ``back-off" to a more expensive state-of-the-art model, leading to a combined system which is both more accurate and 2x faster than the state-of-the-art model alone.

Key facts. Cited by 198 papers in the map (data through 7 Oct 2026). Builds on 50 of its 72 references that are in the map. First posted on arXiv 13 Feb 2021.

Builds on

50 of 72 references.

References · 50

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  5. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  6. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  7. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  8. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  9. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  10. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
  11. 2017Billion-scale similarity search with GPUscited by 2,477
  12. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,201
38 more
  1. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
  2. 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,538
  3. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,486
  4. 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,203
  5. 2019How Can We Know What Language Models Know?cited by 953
  6. 2016Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphscited by 853
  7. 2017Reading Wikipedia to Answer Open-Domain Questionscited by 777
  8. 2020How Much Knowledge Can You Pack Into the Parameters of a Language Model?cited by 713
  9. 2019Latent Retrieval for Weakly Supervised Open Domain Question Answeringcited by 691
  10. 2020KILT: a Benchmark for Knowledge Intensive Language Taskscited by 571
  11. 2020AmbigQA: Answering Ambiguous Open-domain Questionscited by 385
  12. 2020Autoregressive Entity Retrievalcited by 333
  13. 2019Document Expansion by Query Predictioncited by 225
  14. 2017Learning to Ask: Neural Question Generation for Reading Comprehensioncited by 186
  15. 2020How Context Affects Language Models' Factual Predictionscited by 181
  16. 2019Synthetic QA Corpora Generation with Roundtrip Consistencycited by 143
  17. 2020Question and Answer Test-Train Overlap in Open-Domain Question Answering Datasetscited by 137
  18. 2020Entities as Experts: Sparse Memory Access with Entity Supervisioncited by 100
  19. 2020Learning Dense Representations of Phrases at Scalecited by 91
  20. 2019A Discrete Hard EM Approach for Weakly Supervised Question Answeringcited by 78
  21. 2019Real-Time Open-Domain Question Answering with Dense-Sparse Phrase Indexcited by 64
  22. 2019Unsupervised Question Answering by Cloze Translationcited by 59
  23. 2020Every Model Learned by Gradient Descent Is Approximately a Kernel Machinecited by 53
  24. 2021NeurIPS 2020 EfficientQA Competition: Systems, Analyses and Lessons Learnedcited by 52
  25. 2018Large-Scale QA-SRL Parsingcited by 45
  26. 2020Challenges in Information-Seeking QA: Unanswerable Questions and Paragraph Retrievalcited by 44
  27. 2020Facts as Experts: Adaptable and Interpretable Neural Memory over Symbolic Knowledgecited by 37
  28. 2019Quizbowl: The Case for Incremental Question Answeringcited by 36
  29. 2016Generating Factoid Questions With Recurrent Neural Networks: The 30M Factoid Question-Answer Corpuscited by 33
  30. 2020A Memory Efficient Baseline for Open Domain Question Answeringcited by 32
  31. 2018Phrase-Indexed Question Answering: A New Challenge for Scalable Document Comprehensioncited by 23
  32. 2019Generalizing Natural Language Analysis through Span-relation Representationscited by 22
  33. 2019Data Augmentation for BERT Fine-Tuning in Open-Domain Question Answeringcited by 21
  34. 2019Evaluating Rewards for Question Generation Modelscited by 15
  35. 2020Accelerating Real-Time Question Answering via Question Generationcited by 11
  36. 2020Studying Strategically: Learning to Mask for Closed-book QAcited by 10
  37. 2017Safer Classification by Synthesiscited by 8
  38. 2020Open-Domain Question Answering with Pre-Constructed Question Spacescited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 34; 2022: 54; 2023: 44; 2024: 30; 2025: 24; 2026: 12.

198 papers.

Sort:

By area:

  1. 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoningcited by 1,449
  2. 2022Precise Zero-Shot Dense Retrieval without Relevance Labelscited by 508
  3. 2024NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Modelscited by 466
  4. 2022One Embedder, Any Task: Instruction-Finetuned Text Embeddingscited by 329
  5. 2022RARR: Researching and Revising What Language Models Say, Using Language Modelscited by 286
  6. 2024Nomic Embed: Training a Reproducible Long Context Text Embeddercited by 255
  7. 2022Promptagator: Few-shot Dense Retrieval From 8 Examplescited by 233
  8. 2024Generative Representational Instruction Tuningcited by 227
  9. 2022MuRAG: Multimodal Retrieval-Augmented Generator for Open Question Answering over Images and Textcited by 216
  10. 2021Towards Continual Knowledge Learning of Language Modelscited by 172
  11. 2024MM-Embed: Universal Multimodal Retrieval with Multimodal LLMscited by 159
  12. 2021Simple Entity-Centric Questions Challenge Dense Retrieverscited by 140
38 more
  1. 2022Improving the Domain Adaptation of Retrieval Augmented Generation (RAG) Models for Open Domain Question Answeringcited by 126
  2. 2021Uni-Perceiver: Pre-training Unified Architecture for Generic Perception for Zero-shot and Few-shot Taskscited by 124
  3. 2022StreamingQA: A Benchmark for Adaptation to New Knowledge over Time in Question Answering Modelscited by 123
  4. 2021GPL: Generative Pseudo Labeling for Unsupervised Domain Adaptation of Dense Retrievalcited by 119
  5. 2022Autoregressive Search Engines: Generating Substrings as Document Identifierscited by 112
  6. 2021Case-based Reasoning for Natural Language Queries over Knowledge Basescited by 107
  7. 2022Task-aware Retrieval with Instructionscited by 107
  8. 2024LongRAG: Enhancing Retrieval-Augmented Generation with Long-context LLMscited by 104
  9. 2021Improving Question Answering Model Robustness with Synthetic Adversarial Data Generationcited by 77
  10. 2024SuRe: Summarizing Retrievals using Answer Candidates for Open-domain QA of LLMscited by 67
  11. 2023A Multi-Task Embedder For Retrieval Augmented LLMscited by 63
  12. 2021The Web Is Your Oyster - Knowledge-Intensive NLP against a Very Large Web Corpuscited by 60
  13. 2021Salient Phrase Aware Dense Retrieval: Can a Dense Retriever Imitate a Sparse One?cited by 59
  14. 2022Dialog Inpainting: Turning Documents into Dialogscited by 56
  15. 2022Generating Data to Mitigate Spurious Correlations in Natural Language Inference Datasetscited by 54
  16. 2021GooAQ: Open Question Answering with Diverse Answer Typescited by 53
  17. 2021Domain-matched Pre-training Tasks for Dense Retrievalcited by 53
  18. 2023Why So Gullible? Enhancing the Robustness of Retrieval-Augmented Models against Counterfactual Noisecited by 52
  19. 2021Multilingual Autoregressive Entity Linkingcited by 50
  20. 2024Re-Invoke: Tool Invocation Rewriting for Zero-Shot Tool Retrievalcited by 50
  21. 2024Fine Tuning vs. Retrieval Augmented Generation for Less Popular Knowledgecited by 49
  22. 2023LongForm: Effective Instruction Tuning with Reverse Instructionscited by 47
  23. 2021Learning to Retrieve Passages without Supervisioncited by 46
  24. 2023Huatuo-26M, a Large-scale Chinese Medical QA Datasetcited by 46
  25. 2025Llama-Embed-Nemotron-8B: A Universal Text Embedding Model for Multilingual and Cross-Lingual Taskscited by 46
  26. 2024ProSparse: Introducing and Enhancing Intrinsic Activation Sparsity within Large Language Modelscited by 43
  27. 2025Ask in Any Modality: A Comprehensive Survey on Multimodal Retrieval-Augmented Generationcited by 42
  28. 2021BERTnesia: Investigating the capture and forgetting of knowledge in BERTcited by 40
  29. 2022Self-Prompting Large Language Models for Zero-Shot Open-Domain QAcited by 39
  30. 2022An Efficient Memory-Augmented Transformer for Knowledge-Intensive NLP Taskscited by 36
  31. 2024CopyBench: Measuring Literal and Non-Literal Reproduction of Copyright-Protected Text in Language Model Generationcited by 35
  32. 2025KaLM-Embedding-V2: Superior Training Techniques and Data Inspire A Versatile Embedding Modelcited by 35
  33. 2021Models in the Loop: Aiding Crowdworkers with Generative Annotation Assistantscited by 31
  34. 2021Challenges in Generalization in Open Domain Question Answeringcited by 30
  35. 2021How BPE Affects Memorization in Transformerscited by 29
  36. 2021Retrieval-guided Counterfactual Generation for QAcited by 28
  37. 2022Generative Language Models for Paragraph-Level Question Generationcited by 28
  38. 2022Named Entity Recognition in Twitter: A Dataset and Analysis on Short-Term Temporal Shiftscited by 27

50 most cited of 198 All 198 (JSON)