Temporal Reasoning on Implicit Events from Distant Supervision
Not on the map. Why?
Abstract (arXiv)
We propose TRACIE, a novel temporal reasoning dataset that evaluates the degree to which systems understand implicit events -- events that are not mentioned explicitly in natural language text but can be inferred from it. This introduces a new challenge in temporal reasoning research, where prior work has focused on explicitly mentioned events. Human readers can infer implicit events via commonsense reasoning, resulting in a more comprehensive understanding of the situation and, consequently, better reasoning about time. We find, however, that state-of-the-art models struggle when predicting temporal relationships between implicit and explicit events. To address this, we propose a neuro-symbolic temporal reasoning model, SYMTIME, which exploits distant supervision signals from large-scale text and uses temporal rules to combine start times and durations to infer end times. SYMTIME outperforms strong baseline systems on TRACIE by 5%, and by 11% in a zero prior knowledge training setting. Our approach also generalizes to other temporal reasoning tasks, as evidenced by a gain of 1%-9% on MATRES, an explicit event benchmark.
Key facts. Cited by 58 papers in the map (data through 14 Oct 2026). First posted on arXiv 24 Oct 2020.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 5; 2022: 15; 2023: 14; 2024: 12; 2025: 10; 2026: 2.
58 papers.
Sort:
By area:
- 2021Time-Aware Language Models as Temporal Knowledge Basescited by 296
- 2023Large-scale Multi-Modal Pre-trained Models: A Comprehensive Surveycited by 153
- 2021SituatedQA: Incorporating Extra-Linguistic Contexts into QAcited by 145
- 2024Large Language Models Can Learn Temporal Reasoningcited by 111
- 2022State-of-the-art generalisation research in NLP: A taxonomy and reviewcited by 98
- 2024Stable LM 2 1.6B Technical Reportcited by 90
- 2024Test of Time: A Benchmark for Evaluating LLMs on Temporal Reasoningcited by 88
- 2021Think you have Solved Direct-Answer Question Answering? Try ARC-DA, the Direct-Answer AI2 Reasoning Challengecited by 80
- 2024A Peek into Token Bias: Large Language Models Are Not Yet Genuine Reasonerscited by 73
- 2023Test of Time: Instilling Video-Language Models with a Sense of Timecited by 51
- 2023Language Models Can Improve Event Prediction by Few-Shot Abductive Reasoningcited by 45
- 2023Benchmarks for Automated Commonsense Reasoning: A Surveycited by 40
38 more
- 2022Is Neuro-Symbolic AI Meeting its Promise in Natural Language Processing? A Structured Reviewcited by 38
- 2022A Survey of Knowledge-Intensive NLP with Pre-Trained Language Modelscited by 30
- 2025Time-R1: Towards Comprehensive Temporal Reasoning in LLMscited by 25
- 2022DQ-BART: Efficient Sequence-to-Sequence Model via Joint Distillation and Quantizationcited by 23
- 2024TimeArena: Shaping Efficient Multitasking Language Agents in a Time-Aware Simulationcited by 23
- 2021VidLanKD: Improving Language Understanding via Video-Distilled Knowledge Transfercited by 22
- 2022There is a Time and Place for Reasoning Beyond the Imagecited by 22
- 2023COLA: Contextualized Commonsense Causal Reasoning from the Causal Inference Perspectivecited by 22
- 2023Open-Domain Hierarchical Event Schema Induction by Incremental Prompting and Verificationcited by 19
- 2022Learning to Decompose: Hypothetical Question Decomposition Based on Comparable Textscited by 18
- 2022Generic Temporal Reasoning with Differential Analysis and Explanationcited by 18
- 2024BIRD: A Trustworthy Bayesian Inference Framework for Large Language Modelscited by 18
- 2025ChronoSense: Exploring Temporal Understanding in Large Language Models with Time Intervals of Eventscited by 18
- 2024UnSeenTimeQA: Time-Sensitive Question-Answering Beyond LLMs' Memorizationcited by 17
- 2021Salience-Aware Event Chain Modeling for Narrative Understandingcited by 15
- 2022Extracting or Guessing? Improving Faithfulness of Event Temporal Relation Extractioncited by 15
- 2025It's High Time: A Survey of Temporal Question Answeringcited by 14
- 2023Getting Sick After Seeing a Doctor? Diagnosing and Mitigating Knowledge Conflicts in Event Temporal Reasoningcited by 13
- 2023An Overview Of Temporal Commonsense Reasoning and Acquisitioncited by 10
- 2024Assessing Episodic Memory in LLMs with Sequence Order Recall Taskscited by 10
- 2025LexTime: A Benchmark for Temporal Ordering of Legal Eventscited by 10
- 2022Zero-Shot On-the-Fly Event Schema Inductioncited by 9
- 2022Synopses of Movie Narratives: a Video-Language Dataset for Story Understandingcited by 8
- 2023EvEval: A Comprehensive Evaluation of Event Semantics for Large Language Modelscited by 6
- 2025Unlocking the Potential of Generative AI through Neuro-Symbolic Architectures: Benefits and Limitationscited by 6
- 2024Towards Effective Time-Aware Language Representation: Exploring Enhanced Temporal Understanding in Language Modelscited by 5
- 2025Counterfactual-Consistency Prompting for Relative Temporal Understanding in Large Language Modelscited by 5
- 2022Learning Action Conditions from Instructional Manuals for Instruction Understandingcited by 4
- 2022A Closer Look at Temporal Ordering in the Segmentation of Instructional Videoscited by 4
- 2023NLP Reproducibility For All: Understanding Experiences of Beginnerscited by 4
- 2024Unveiling Divergent Inductive Biases of LLMs on Temporal Datacited by 4
- 2024A Comprehensive Evaluation on Event Reasoning of Large Language Modelscited by 4
- 2022Some Reflections on Drawing Causal Inference using Textual Data: Parallels Between Human Subjects and Organized Textscited by 2
- 2022What Has Been Enhanced in my Knowledge-Enhanced Language Model?cited by 2
- 2023Large Language Models as Sous Chefs: Revising Recipes with GPT-3cited by 2
- 2023Choice-75: A Dataset on Decision Branching in Script Learningcited by 2
- 2023From Heuristic to Analytic: Cognitively Motivated Strategies for Coherent Physical Commonsense Reasoningcited by 2
- 2024Temporal Validity Change Predictioncited by 2
50 most cited of 58 All 58 (JSON)
- 2026Temporal Reasoning Is Not the Bottleneck: A Probabilistic Inconsistency Framework for Neuro-Symbolic QA5 May 2026
- 2026Looking for the Bottleneck in Fine-grained Temporal Relation Classification27 Apr 2026
- 2025Everyday Physics in Korean Contexts: A Culturally Grounded Physical Reasoning Benchmark22 Sep 2025
- 2025MobQA: A Benchmark Dataset for Semantic Understanding of Human Mobility Data through Question Answering15 Aug 2025
- 2025Discrete Minds in a Continuous World: Do Language Models Know Time Passes?6 Jun 2025
- 2025LexTime: A Benchmark for Temporal Ordering of Legal Events4 Jun 2025
- 2025FABLE: A Novel Data-Flow Analysis Benchmark on Procedural Text for Large Language Model Evaluation30 May 2025
- 2025It's High Time: A Survey of Temporal Question Answering26 May 2025
- 2025Time-R1: Towards Comprehensive Temporal Reasoning in LLMs16 May 2025
- 2025Counterfactual-Consistency Prompting for Relative Temporal Understanding in Large Language Models17 Feb 2025
- 2025Unlocking the Potential of Generative AI through Neuro-Symbolic Architectures: Benefits and Limitations16 Feb 2025
- 2025ChronoSense: Exploring Temporal Understanding in Large Language Models with Time Intervals of Events6 Jan 2025
38 more
- 2024Assessing Episodic Memory in LLMs with Sequence Order Recall Tasks10 Oct 2024
- 2024UnSeenTimeQA: Time-Sensitive Question-Answering Beyond LLMs' Memorization3 Jul 2024
- 2024A Peek into Token Bias: Large Language Models Are Not Yet Genuine Reasoners16 Jun 2024
- 2024Test of Time: A Benchmark for Evaluating LLMs on Temporal Reasoning13 Jun 2024
- 2024Towards Effective Time-Aware Language Representation: Exploring Enhanced Temporal Understanding in Language Models4 Jun 2024
- 2024A Comprehensive Evaluation on Event Reasoning of Large Language Models26 Apr 2024
- 2024BIRD: A Trustworthy Bayesian Inference Framework for Large Language Models18 Apr 2024
- 2024Unveiling Divergent Inductive Biases of LLMs on Temporal Data1 Apr 2024
- 2024Stable LM 2 1.6B Technical Report27 Feb 2024
- 2024TimeArena: Shaping Efficient Multitasking Language Agents in a Time-Aware Simulation8 Feb 2024
- 2024Large Language Models Can Learn Temporal Reasoning12 Jan 2024
- 2024Temporal Validity Change Prediction1 Jan 2024
- 2023From Heuristic to Analytic: Cognitively Motivated Strategies for Coherent Physical Commonsense Reasoning24 Oct 2023
- 2023Choice-75: A Dataset on Decision Branching in Script Learning21 Sep 2023
- 2023An Overview Of Temporal Commonsense Reasoning and Acquisition28 Jul 2023
- 2023Open-Domain Hierarchical Event Schema Induction by Incremental Prompting and Verification5 Jul 2023
- 2023Large Language Models as Sous Chefs: Revising Recipes with GPT-324 Jun 2023
- 2023Language Models Can Improve Event Prediction by Few-Shot Abductive Reasoning26 May 2023
- 2023NLP Reproducibility For All: Understanding Experiences of Beginners26 May 2023
- 2023EvEval: A Comprehensive Evaluation of Event Semantics for Large Language Models24 May 2023
- 2023Getting Sick After Seeing a Doctor? Diagnosing and Mitigating Knowledge Conflicts in Event Temporal Reasoning24 May 2023
- 2023COLA: Contextualized Commonsense Causal Reasoning from the Causal Inference Perspective9 May 2023
- 2023Causal schema induction for knowledge discovery27 Mar 2023
- 2023Large-scale Multi-Modal Pre-trained Models: A Comprehensive Survey20 Feb 2023
- 2023Benchmarks for Automated Commonsense Reasoning: A Survey9 Feb 2023
- 2023Test of Time: Instilling Video-Language Models with a Sense of Time5 Jan 2023
- 2022Generic Temporal Reasoning with Differential Analysis and Explanation20 Dec 2022
- 2022Learning to Decompose: Hypothetical Question Decomposition Based on Comparable Texts30 Oct 2022
- 2022Zero-Shot On-the-Fly Event Schema Induction12 Oct 2022
- 2022Cross-Lingual Speaker Identification Using Distant Supervision11 Oct 2022
- 2022Extracting or Guessing? Improving Faithfulness of Event Temporal Relation Extraction10 Oct 2022
- 2022State-of-the-art generalisation research in NLP: A taxonomy and review6 Oct 2022
- 2022A Closer Look at Temporal Ordering in the Segmentation of Instructional Videos30 Sep 2022
- 2022Learning Action Conditions from Instructional Manuals for Instruction Understanding25 May 2022
- 2022DQ-BART: Efficient Sequence-to-Sequence Model via Joint Distillation and Quantization21 Mar 2022
- 2022Synopses of Movie Narratives: a Video-Language Dataset for Story Understanding11 Mar 2022
- 2022There is a Time and Place for Reasoning Beyond the Image1 Mar 2022
- 2022Is Neuro-Symbolic AI Meeting its Promise in Natural Language Processing? A Structured Review24 Feb 2022
50 newest of 58 All 58 (JSON)
- 2024Large Language Models Can Learn Temporal Reasoningcited by 111
- 2024Test of Time: A Benchmark for Evaluating LLMs on Temporal Reasoningcited by 88
- 2024A Peek into Token Bias: Large Language Models Are Not Yet Genuine Reasonerscited by 73
- 2023Language Models Can Improve Event Prediction by Few-Shot Abductive Reasoningcited by 45
- 2023Benchmarks for Automated Commonsense Reasoning: A Surveycited by 40
- 2022Is Neuro-Symbolic AI Meeting its Promise in Natural Language Processing? A Structured Reviewcited by 38
- 2025Time-R1: Towards Comprehensive Temporal Reasoning in LLMscited by 25
- 2023COLA: Contextualized Commonsense Causal Reasoning from the Causal Inference Perspectivecited by 22
- 2022Generic Temporal Reasoning with Differential Analysis and Explanationcited by 18
- 2023Getting Sick After Seeing a Doctor? Diagnosing and Mitigating Knowledge Conflicts in Event Temporal Reasoningcited by 13
- 2023An Overview Of Temporal Commonsense Reasoning and Acquisitioncited by 10
- 2024Towards Effective Time-Aware Language Representation: Exploring Enhanced Temporal Understanding in Language Modelscited by 5
12 most cited of 18 All 18 (JSON)
- 2026Temporal Reasoning Is Not the Bottleneck: A Probabilistic Inconsistency Framework for Neuro-Symbolic QA5 May 2026
- 2025Everyday Physics in Korean Contexts: A Culturally Grounded Physical Reasoning Benchmark22 Sep 2025
- 2025Discrete Minds in a Continuous World: Do Language Models Know Time Passes?6 Jun 2025
- 2025Time-R1: Towards Comprehensive Temporal Reasoning in LLMs16 May 2025
- 2025Counterfactual-Consistency Prompting for Relative Temporal Understanding in Large Language Models17 Feb 2025
- 2024A Peek into Token Bias: Large Language Models Are Not Yet Genuine Reasoners16 Jun 2024
- 2024Test of Time: A Benchmark for Evaluating LLMs on Temporal Reasoning13 Jun 2024
- 2024Towards Effective Time-Aware Language Representation: Exploring Enhanced Temporal Understanding in Language Models4 Jun 2024
- 2024A Comprehensive Evaluation on Event Reasoning of Large Language Models26 Apr 2024
- 2024Large Language Models Can Learn Temporal Reasoning12 Jan 2024
- 2023From Heuristic to Analytic: Cognitively Motivated Strategies for Coherent Physical Commonsense Reasoning24 Oct 2023
- 2023An Overview Of Temporal Commonsense Reasoning and Acquisition28 Jul 2023
12 newest of 18 All 18 (JSON)
- 2021Time-Aware Language Models as Temporal Knowledge Basescited by 296
- 2023Large-scale Multi-Modal Pre-trained Models: A Comprehensive Surveycited by 153
- 2021SituatedQA: Incorporating Extra-Linguistic Contexts into QAcited by 145
- 2022State-of-the-art generalisation research in NLP: A taxonomy and reviewcited by 98
- 2024Stable LM 2 1.6B Technical Reportcited by 90
- 2021Think you have Solved Direct-Answer Question Answering? Try ARC-DA, the Direct-Answer AI2 Reasoning Challengecited by 80
- 2023Test of Time: Instilling Video-Language Models with a Sense of Timecited by 51
- 2022A Survey of Knowledge-Intensive NLP with Pre-Trained Language Modelscited by 30
- 2022DQ-BART: Efficient Sequence-to-Sequence Model via Joint Distillation and Quantizationcited by 23
- 2022There is a Time and Place for Reasoning Beyond the Imagecited by 22
- 2023Open-Domain Hierarchical Event Schema Induction by Incremental Prompting and Verificationcited by 19
- 2022Learning to Decompose: Hypothetical Question Decomposition Based on Comparable Textscited by 18
12 most cited of 38 All 38 (JSON)
- 2026Looking for the Bottleneck in Fine-grained Temporal Relation Classification27 Apr 2026
- 2025MobQA: A Benchmark Dataset for Semantic Understanding of Human Mobility Data through Question Answering15 Aug 2025
- 2025LexTime: A Benchmark for Temporal Ordering of Legal Events4 Jun 2025
- 2025FABLE: A Novel Data-Flow Analysis Benchmark on Procedural Text for Large Language Model Evaluation30 May 2025
- 2025It's High Time: A Survey of Temporal Question Answering26 May 2025
- 2025Unlocking the Potential of Generative AI through Neuro-Symbolic Architectures: Benefits and Limitations16 Feb 2025
- 2025ChronoSense: Exploring Temporal Understanding in Large Language Models with Time Intervals of Events6 Jan 2025
- 2024Assessing Episodic Memory in LLMs with Sequence Order Recall Tasks10 Oct 2024
- 2024UnSeenTimeQA: Time-Sensitive Question-Answering Beyond LLMs' Memorization3 Jul 2024
- 2024BIRD: A Trustworthy Bayesian Inference Framework for Large Language Models18 Apr 2024
- 2024Unveiling Divergent Inductive Biases of LLMs on Temporal Data1 Apr 2024
- 2024Stable LM 2 1.6B Technical Report27 Feb 2024
12 newest of 38 All 38 (JSON)