mlsanityData to 7 Oct 2026

RealTime QA: What's the Answer Right Now?

arXiv 2207.13332 ↗First posted 27 Jul 2022cs.CLcited by 239

Not on the map. Why?

Cited by 59 in Retrieval-augmented generation, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We introduce REALTIME QA, a dynamic question answering (QA) platform that announces questions and evaluates systems on a regular basis (weekly in this version). REALTIME QA inquires about the current world, and QA systems need to answer questions about novel events or information. It therefore challenges static, conventional assumptions in open-domain QA datasets and pursues instantaneous applications. We build strong baseline models upon large pretrained language models, including GPT-3 and T5. Our benchmark is an ongoing effort, and this paper presents real-time evaluation results over the past year. Our experimental results show that GPT-3 can often properly update its generation results, based on newly-retrieved documents, highlighting the importance of up-to-date information retrieval. Nonetheless, we find that GPT-3 tends to return outdated answers when retrieved documents do not provide sufficient information to find an answer. This suggests an important avenue for future research: can an open-domain QA system identify such unanswerable cases and communicate with the user or even the retrieval module to modify the retrieval results? We hope that REALTIME QA will spur progress in instantaneous applications of question answering and beyond.

Key facts. Cited by 239 papers in the map (data through 7 Oct 2026). Builds on 56 of its 99 references that are in the map. First posted on arXiv 27 Jul 2022.

Builds on

56 of 99 references.

References · 56

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  5. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  6. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  7. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
  8. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  9. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  10. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  11. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
  12. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,793
38 more
  1. 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,538
  2. 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,203
  3. 2015Teaching Machines to Read and Comprehendcited by 1,039
  4. 2017RACE: Large-scale ReAding Comprehension Dataset From Examinationscited by 851
  5. 2017Reading Wikipedia to Answer Open-Domain Questionscited by 777
  6. 2020How Much Knowledge Can You Pack Into the Parameters of a Language Model?cited by 713
  7. 2020TyDi QA: A Benchmark for Information-Seeking Question Answering in Typologically Diverse Languagescited by 486
  8. 2021Dynabench: Rethinking Benchmarking in NLPcited by 447
  9. 2020AmbigQA: Answering Ambiguous Open-domain Questionscited by 385
  10. 2021Entity-Based Knowledge Conflicts in Question Answeringcited by 349
  11. 2018SWAG: A Large-Scale Adversarial Dataset for Grounded Commonsense Inferencecited by 333
  12. 2021Time-Aware Language Models as Temporal Knowledge Basescited by 290
  13. 2021The GEM Benchmark: Natural Language Generation, its Evaluation and Metricscited by 239
  14. 2021Mind the Gap: Assessing Temporal Generalization in Neural Language Modelscited by 226
  15. 2020CORD-19: The COVID-19 Open Research Datasetcited by 179
  16. 2021Towards Continual Knowledge Learning of Language Modelscited by 172
  17. 2020MKQA: A Linguistically Diverse Benchmark for Multilingual Open Domain Question Answeringcited by 151
  18. 2021A Dataset for Answering Time-Sensitive Questionscited by 150
  19. 2021SituatedQA: Incorporating Extra-Linguistic Contexts into QAcited by 145
  20. 2022StreamingQA: A Benchmark for Adaptation to New Knowledge over Time in Question Answering Modelscited by 123
  21. 2020XOR QA: Cross-lingual Open-Retrieval Question Answeringcited by 121
  22. 2021Mr. TyDi: A Multi-lingual Benchmark for Dense Retrievalcited by 118
  23. 2022TemporalWiki: A Lifelong Benchmark for Training and Evaluating Ever-Evolving Language Modelscited by 117
  24. 2019Multi-passage BERT: A Globally Normalized BERT Model for Open-domain Question Answeringcited by 96
  25. 2021Time Waits for No One! Analysis and Challenges of Temporal Misalignmentcited by 91
  26. 2019Real-Time Open-Domain Question Answering with Dense-Sparse Phrase Indexcited by 64
  27. 2020DynaSent: A Dynamic Benchmark for Sentiment Analysiscited by 63
  28. 2019Knowledge Guided Text Retrieval and Reading for Open Domain Question Answeringcited by 59
  29. 2022Entity Cloze By Date: What LMs Know About Unseen Entitiescited by 55
  30. 2021Dynaboard: An Evaluation-As-A-Service Platform for Holistic Next-Generation Benchmarkingcited by 53
  31. 2021Temporal Adaptation of BERT and Performance on Downstream Document Classification: Insights from Social Mediacited by 51
  32. 2014Processing Social Media Messages in Mass Emergency: A Surveycited by 50
  33. 2016Twitter as a Lifeline: Human-annotated Twitter Corpora for NLP of Crisis-related Messagescited by 43
  34. 2022GEMv2: Multilingual NLG Benchmarking in a Single Line of Codecited by 36
  35. 2021Bidimensional Leaderboards: Generate and Evaluate Language Hand in Handcited by 32
  36. 2020Examining the State-of-the-Art in News Timeline Summarizationcited by 27
  37. 2021How Robust are Model Rankings: A Leaderboard Customization Approach for Equitable Evaluationcited by 24
  38. 2022Plug-and-Play Adaptation for Continuously-updated QAcited by 21

50 most cited of 56 All 56 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 5; 2023: 32; 2024: 86; 2025: 82; 2026: 34.

239 papers.

Sort:

By area:

  1. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,582
  2. 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,103
  3. 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,054
  4. 2022Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questionscited by 911
  5. 2024Adaptive-RAG: Learning to Adapt Retrieval-Augmented Large Language Models through Question Complexitycited by 437
  6. 2023FreshLLMs: Refreshing Large Language Models with Search Engine Augmentationcited by 308
  7. 2023Tool Learning with Foundation Modelscited by 302
  8. 2024Knowledge Conflicts for LLMs: A Surveycited by 274
  9. 2023ToolQA: A Dataset for LLM Question Answering with External Toolscited by 273
  10. 2022Prompting GPT-3 To Be Reliablecited by 258
  11. 2023Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificitycited by 204
  12. 2024RankRAG: Unifying Context Ranking with Retrieval-Augmented Generation in LLMscited by 200
38 more
  1. 2023FLASK: Fine-grained Language Model Evaluation based on Alignment Skill Setscited by 198
  2. 2023Chain-of-Note: Enhancing Robustness in Retrieval-Augmented Language Modelscited by 180
  3. 2024Don't Hallucinate, Abstain: Identifying LLM Knowledge Gaps via Multi-LLM Collaborationcited by 176
  4. 2024Fact, Fetch, and Reason: A Unified Evaluation of Retrieval-Augmented Generationcited by 158
  5. 2022Ask Me Anything: A simple strategy for prompting language modelscited by 155
  6. 2024Evaluation of Retrieval-Augmented Generation: A Surveycited by 146
  7. 2024Certifiably Robust RAG against Retrieval Corruptioncited by 144
  8. 2024Know Your Limits: A Survey of Abstention in Large Language Modelscited by 116
  9. 2024InstructRAG: Instructing Retrieval-Augmented Generation via Self-Synthesized Rationalescited by 100
  10. 2024From Matching to Generation: A Survey on Generative Information Retrievalcited by 89
  11. 2024Tuning Language Models by Proxycited by 88
  12. 2025HalluLens: LLM Hallucination Benchmarkcited by 88
  13. 2023KoLA: Carefully Benchmarking World Knowledge of Large Language Modelscited by 85
  14. 2024Simple Is Effective: The Roles of Graphs and Large Language Models in Knowledge-Graph-Based Retrieval-Augmented Generationcited by 78
  15. 2023Context-faithful Prompting for Large Language Modelscited by 76
  16. 2024Reliable, Adaptable, and Attributable Language Models with Retrievalcited by 73
  17. 2025SealQA: Raising the Bar for Reasoning in Search-Augmented Language Modelscited by 65
  18. 2024Typos that Broke the RAG's Back: Genetic Attack on RAG Pipeline by Simulating Documents in the Wild via Low-level Perturbationscited by 64
  19. 2023Evaluating GPT-4 and ChatGPT on Japanese Medical Licensing Examinationscited by 63
  20. 2025Retrieval-Augmented Generation with Conflicting Evidencecited by 60
  21. 2024Knowledge Mechanisms in Large Language Models: A Survey and Perspectivecited by 50
  22. 2024Fine Tuning vs. Retrieval Augmented Generation for Less Popular Knowledgecited by 49
  23. 2024Enhancing Contextual Understanding in Large Language Models through Contrastive Decodingcited by 49
  24. 2025LLM-based Agents Suffer from Hallucinations: A Survey of Taxonomy, Methods, and Directionscited by 49
  25. 2024Dated Data: Tracing Knowledge Cutoffs in Large Language Modelscited by 48
  26. 2023Knowledge Card: Filling LLMs' Knowledge Gaps with Plug-in Specialized Language Modelscited by 46
  27. 2025Large Language Models Hallucination: A Comprehensive Surveycited by 45
  28. 2025SimpleQA Verified: A Reliable Factuality Benchmark to Measure Parametric Knowledgecited by 44
  29. 2026DeepSearchQA: Bridging the Comprehensiveness Gap for Deep Research Agentscited by 44
  30. 2024AntiLeakBench: Preventing Data Contamination by Automatically Constructing Benchmarks with Updated Real-World Knowledgecited by 42
  31. 2024What is the Role of Small Models in the LLM Era: A Surveycited by 41
  32. 2024Rationale-Guided Retrieval Augmented Generation for Medical Question Answeringcited by 41
  33. 2023Towards Benchmarking and Improving the Temporal Reasoning Capability of Large Language Modelscited by 37
  34. 2023Temporal Knowledge Graph Forecasting Without Knowledge Using In-Context Learningcited by 36
  35. 2024Knowledge Boundary of Large Language Models: A Surveycited by 36
  36. 2024MemLLM: Finetuning LLMs to Use An Explicit Read-Write Memorycited by 35
  37. 2024AdaCAD: Adaptively Decoding to Balance Conflicts between Contextual and Parametric Knowledgecited by 35
  38. 2023How Do Large Language Models Capture the Ever-changing World Knowledge? A Review of Recent Advancescited by 33

50 most cited of 239 All 239 (JSON)