mlsanityData to 7 Oct 2026

A Dataset for Answering Time-Sensitive Questions

arXiv 2108.06314 ↗First posted 13 Aug 2021cs.CL cs.AIcited by 150

Not on the map. Why?

Cited by 30 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Time is an important dimension in our physical world. Lots of facts can evolve with respect to time. For example, the U.S. President might change every four years. Therefore, it is important to consider the time dimension and empower the existing QA models to reason over time. However, the existing QA datasets contain rather few time-sensitive questions, hence not suitable for diagnosing or benchmarking the model's temporal reasoning capability. In order to promote research in this direction, we propose to construct a time-sensitive QA dataset. The dataset is constructed by 1) mining time-evolving facts from WikiData and aligning them to their corresponding Wikipedia page, 2) employing crowd workers to verify and calibrate these noisy facts, 3) generating question-answer pairs based on the annotated time-sensitive facts. Our dataset poses challenges in the aspect of both temporal understanding and temporal reasoning. We evaluate different SoTA long-document QA systems like BigBird and FiD on our dataset. The best-performing model FiD can only achieve 46% accuracy, still far behind the human performance of 87%. We demonstrate that these models are still lacking the ability to perform consistent temporal reasoning. Therefore, we believe that our dataset could serve as a benchmark to develop NLP models more sensitive to temporal shifts. The dataset and code are released in~urlhttps://github.com/wenhuchen/Time-Sensitive-QA.

Key facts. Cited by 150 papers in the map (data through 7 Oct 2026). Builds on 22 of its 45 references that are in the map. First posted on arXiv 13 Aug 2021.

Builds on

22 of 45 references.

References · 22

  1. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  2. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  3. 2020Longformer: The Long-Document Transformercited by 3,362
  4. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  5. 2020Big Bird: Transformers for Longer Sequencescited by 1,673
  6. 2020Reformer: The Efficient Transformercited by 1,658
  7. 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,538
  8. 2019Generating Long Sequences with Sparse Transformerscited by 1,523
  9. 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,203
  10. 2019DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphscited by 883
  11. 2017Reading Wikipedia to Answer Open-Domain Questionscited by 777
  12. 2020Long Range Arena: A Benchmark for Efficient Transformerscited by 696
10 more
  1. 2020Efficient Content-Based Sparse Attention with Routing Transformerscited by 469
  2. 2021Time-Aware Language Models as Temporal Knowledge Basescited by 290
  3. 2017SearchQA: A New Q&A Dataset Augmented with Context from a Search Enginecited by 221
  4. 2021Question Answering Over Temporal Knowledge Graphscited by 99
  5. 2020TORQUE: A Reading Comprehension Dataset of Temporal Ordering Questionscited by 65
  6. 2019TEQUILA: Temporal Question Answering over Knowledge Basescited by 47
  7. 2020Answering Open-Domain Questions of Varying Reasoning Steps from Textcited by 43
  8. 2020ETC: Encoding Long and Structured Inputs in Transformerscited by 34
  9. 2021Multi-Step Reasoning Over Unstructured Text with Beam Dense Retrievalcited by 25
  10. 2019CogCompTime: A Tool for Understanding Time in Natural Language Textcited by 23

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 3; 2022: 8; 2023: 20; 2024: 36; 2025: 52; 2026: 31.

150 papers.

Sort:

By area:

  1. 2023FreshLLMs: Refreshing Large Language Models with Search Engine Augmentationcited by 308
  2. 2024Knowledge Conflicts for LLMs: A Surveycited by 274
  3. 2023ToolQA: A Dataset for LLM Question Answering with External Toolscited by 273
  4. 2022Prompting GPT-3 To Be Reliablecited by 258
  5. 2022RealTime QA: What's the Answer Right Now?cited by 239
  6. 2022PromptCast: A New Prompt-based Learning Paradigm for Time Series Forecastingcited by 170
  7. 2023Take a Step Back: Evoking Reasoning via Abstraction in Large Language Modelscited by 157
  8. 2021SituatedQA: Incorporating Extra-Linguistic Contexts into QAcited by 145
  9. 2022StreamingQA: A Benchmark for Adaptation to New Knowledge over Time in Question Answering Modelscited by 123
  10. 2022TemporalWiki: A Lifelong Benchmark for Training and Evaluating Ever-Evolving Language Modelscited by 117
  11. 2024Large Language Models Can Learn Temporal Reasoningcited by 110
  12. 2024Test of Time: A Benchmark for Evaluating LLMs on Temporal Reasoningcited by 83
38 more
  1. 2025SealQA: Raising the Bar for Reasoning in Search-Augmented Language Modelscited by 65
  2. 2022Forecasting Future World Events with Neural Networkscited by 63
  3. 2024Evaluating LLMs at Detecting Errors in LLM Responsescited by 49
  4. 2023TRAM: Benchmarking Temporal Reasoning for Large Language Modelscited by 39
  5. 2023Towards Benchmarking and Improving the Temporal Reasoning Capability of Large Language Modelscited by 37
  6. 2021ConditionalQA: A Complex Reading Comprehension Dataset with Conditional Answerscited by 29
  7. 2023TempTabQA: Temporal Question Answering for Semi-Structured Tablescited by 27
  8. 2021TempoQR: Temporal Question Reasoning over Knowledge Graphscited by 26
  9. 2023IfQA: A Dataset for Open-domain Question Answering under Counterfactual Presuppositionscited by 25
  10. 2024Is Your LLM Outdated? A Deep Look at Temporal Generalizationcited by 25
  11. 2024Set the Clock: Temporal Alignment of Pretrained Language Modelscited by 24
  12. 2024DyKnow: Dynamically Verifying Time-Sensitive Factual Knowledge in LLMscited by 23
  13. 2024Let LLMs Take on the Latest Challenges! A Chinese Dynamic Question Answering Benchmarkcited by 22
  14. 2023Mitigating Temporal Misalignment by Discarding Outdated Factscited by 21
  15. 2024TimeChara: Evaluating Point-in-Time Character Hallucination of Role-Playing Large Language Modelscited by 21
  16. 2026REMem: Reasoning with Episodic Memory in Language Agentcited by 20
  17. 2024Are LLMs Prescient? A Continuous Evaluation using Daily News as the Oraclecited by 19
  18. 2024Two-stage Generative Question Answering on Temporal Knowledge Graph Using Large Language Modelscited by 18
  19. 2024UnSeenTimeQA: Time-Sensitive Question-Answering Beyond LLMs' Memorizationcited by 17
  20. 2024MRAG: A Modular Retrieval Framework for Time-Sensitive Question Answeringcited by 17
  21. 2025ChronoSense: Exploring Temporal Understanding in Large Language Models with Time Intervals of Eventscited by 17
  22. 2022Data Representativity for Machine Learning and AI Systemscited by 15
  23. 2023MenatQA: A New Dataset for Testing the Temporal Comprehension and Reasoning Abilities of Large Language Modelscited by 15
  24. 2024PAT-Questions: A Self-Updating Benchmark for Present-Anchored Temporal Question-Answeringcited by 15
  25. 2023Salient Span Masking for Temporal Understandingcited by 14
  26. 2023Question Answering as Programming for Solving Time-Sensitive Questionscited by 14
  27. 2025Episodic Memories Generation and Evaluation Benchmark for Large Language Modelscited by 14
  28. 2025Does Time Have Its Place? Temporal Heads: Where Language Models Recall Time-specific Informationcited by 14
  29. 2025TempRetriever: Fusion-based Temporal Dense Passage Retrieval for Time-Sensitive Questionscited by 14
  30. 2024Faithful Temporal Question Answering over Heterogeneous Sourcescited by 13
  31. 2025HoH: A Dynamic Benchmark for Evaluating the Impact of Outdated Information on Retrieval-Augmented Generationcited by 13
  32. 2024LeKUBE: A Legal Knowledge Update BEnchmarkcited by 12
  33. 2024EvoWiki: Evaluating LLMs on Evolving Knowledgecited by 12
  34. 2025It's High Time: A Survey of Temporal Question Answeringcited by 12
  35. 2023An Overview Of Temporal Commonsense Reasoning and Acquisitioncited by 10
  36. 2023Once Upon a textitTime in textitGraph: Relative-Time Pretraining for Complex Temporal Reasoningcited by 10
  37. 2025LLMs as Repositories of Factual Knowledge: Limitations and Solutionscited by 10
  38. 2025A Study into Investigating Temporal Robustness of LLMscited by 10

50 most cited of 150 All 150 (JSON)