# Probing Language Models for Understanding of Temporal Expressions

arXiv: https://arxiv.org/abs/2110.01113 · HTML: https://mlsanity.com/paper/2110.01113/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 27 references that are in the map.
- First posted on arXiv 3 Oct 2021.

## Builds on (most cited first, 13 of 13)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference](https://mlsanity.com/paper/1704.05426/) (2017)
- [DeBERTa: Decoding-enhanced BERT with Disentangled Attention](https://mlsanity.com/paper/2006.03654/) (2020)
- [Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inference](https://mlsanity.com/paper/1902.01007/) (2019)
- [A Primer in BERTology: What we know about how BERT works](https://mlsanity.com/paper/2002.12327/) (2020)
- [Hypothesis Only Baselines in Natural Language Inference](https://mlsanity.com/paper/1805.01042/) (2018)
- [Do NLP Models Know Numbers? Probing Numeracy in Embeddings](https://mlsanity.com/paper/1909.07940/) (2019)
- ["Going on a vacation" takes longer than "Going for a walk": A Study of Temporal Commonsense Understanding](https://mlsanity.com/paper/1909.03065/) (2019)
- [TORQUE: A Reading Comprehension Dataset of Temporal Ordering Questions](https://mlsanity.com/paper/2005.00242/) (2020)
- [CogCompTime: A Tool for Understanding Time in Natural Language Text](https://mlsanity.com/paper/1906.04940/) (2019)
- [Fine-Grained Temporal Relation Extraction](https://mlsanity.com/paper/1902.01390/) (2019)
- [Temporal Information Extraction by Predicting Relative Time-lines](https://mlsanity.com/paper/1808.09401/) (2018)

## Built on by (most cited first, 16 of 16)

- [Towards Faithful Model Explanation in NLP: A Survey](https://mlsanity.com/paper/2209.11326/) (2022)
- [Test of Time: Instilling Video-Language Models with a Sense of Time](https://mlsanity.com/paper/2301.02074/) (2023)
- [Limits of Deep Learning: Sequence Modeling through the Lens of Complexity Theory](https://mlsanity.com/paper/2405.16674/) (2024)
- [UnSeenTimeQA: Time-Sensitive Question-Answering Beyond LLMs' Memorization](https://mlsanity.com/paper/2407.03525/) (2024)
- [Salient Span Masking for Temporal Understanding](https://mlsanity.com/paper/2303.12860/) (2023)
- [LexTime: A Benchmark for Temporal Ordering of Legal Events](https://mlsanity.com/paper/2506.04041/) (2025)
- [Representation Of Lexical Stylistic Features In Language Models' Embedding Space](https://mlsanity.com/paper/2305.18657/) (2023)
- [Around the World in 24 Hours: Probing LLM Knowledge of Time and Place](https://mlsanity.com/paper/2506.03984/) (2025)
- [Counterfactual-Consistency Prompting for Relative Temporal Understanding in Large Language Models](https://mlsanity.com/paper/2502.11425/) (2025)
- [Jamp: Controlled Japanese Temporal Inference Dataset for Evaluating Generalization Capacity of Language Models](https://mlsanity.com/paper/2306.10727/) (2023)
- [Compositional Semantics and Inference System for Temporal Order based on Japanese CCG](https://mlsanity.com/paper/2204.09245/) (2022)
- [Pragmatic Reasoning Unlocks Quantifier Semantics for Foundation Models](https://mlsanity.com/paper/2311.04659/) (2023)
- [Discrete Minds in a Continuous World: Do Language Models Know Time Passes?](https://mlsanity.com/paper/2506.05790/) (2025)
- [Towards Probing Contact Center Large Language Models](https://mlsanity.com/paper/2312.15922/) (2023)
- [TempViz: On the Evaluation of Temporal Knowledge in Text-to-Image Models](https://mlsanity.com/paper/2601.14951/) (2026)
- [A Study of Temporal Fusion Strategies for Named Entity Recognition in Historical Texts](https://mlsanity.com/paper/2606.27881/) (2026)

## Abstract (arXiv)

We present three Natural Language Inference (NLI) challenge sets that can evaluate NLI models on their understanding of temporal expressions. More specifically, we probe these models for three temporal properties: (a) the order between points in time, (b) the duration between two points in time, (c) the relation between the magnitude of times specified in different units. We find that although large language models fine-tuned on MNLI have some basic perception of the order between points in time, at large, these models do not have a thorough understanding of the relation between temporal expressions.