Do Large Language Models (LLMs) Understand Chronology?
Not on the map. Why?
Abstract (arXiv)
Large language models (LLMs) are increasingly used in finance and economics, where prompt-based attempts against look-ahead bias implicitly assume that models understand chronology. We test this fundamental question with a series of chronological ordering tasks with increasing complexities over facts the model already knows from pre-training. Our tasks cover (1) chronological ordering, (2) conditional sorting (filter, then order), and (3) anachronism detection. We evaluate GPT-4.1, Claude-3.7 Sonnet, with and without Extended Thinking (ET), and GPT-5 across multiple reasoning-effort settings. Across models, Exact match rate drops sharply as sequences lengthen even while rank correlations stay high as LLMs largely preserve local order but struggle to maintain a single globally consistent timeline. In conditional sorting, most failures stem from the filtering step rather than the ordering step, but GPT-5 and Claude-3.7 Sonnet with Extended Thinking outshine normal models significantly. Lastly, anachronism detection is found to be the easiest task for the LLMs but performance still declines with increasingly overlapping timelines or entities. Overall, our main contribution is showing that allocating explicit reasoning budget helps with chronological ordering with GPT-5 at medium/high reasoning effort achieving flawless ordering at all lengths and perfect conditional sorting (both self-filtered and given-subset), whereas low/minimal effort degrades with longer lists, mirroring earlier models. Our findings delineate limits of current LLMs on chronological tasks, providing insights into task complexity, and demonstrate scenarios in which reasoning helps. These patterns are important for the real-time application of LLMs in finance. We release all code and evaluation templates to support full reproducibility.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 8 of its 26 references that are in the map. First posted on arXiv 18 Nov 2025.
Builds on
8 of 26 references.
References · 8
- 2015Towards AI-Complete Question Answering: A Set of Prerequisite Toy Taskscited by 314
- 2025The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexitycited by 302
- 2021A Dataset for Answering Time-Sensitive Questionscited by 154
- 2018ListOps: A Diagnostic Dataset for Latent Tree Learningcited by 129
- 2023Can ChatGPT Forecast Stock Price Movements? Return Predictability and Large Language Modelscited by 129
- 2024Approaching Human-Level Forecasting with Language Modelscited by 107
- 2023TRAM: Benchmarking Temporal Reasoning for Large Language Modelscited by 40
- 2023Assessing Look-Ahead Bias in Stock Return Predictions Generated By GPT Sentiment Analysiscited by 31
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2024Remember This Event That Year? Assessing Temporal Information and Reasoning in Large Language Models
- 2025Temporal Referential Consistency: Do LLMs Favor Sequences Over Absolute Time References?
- 2023Are Large Language Models Temporally Grounded?
- 2024Time Awareness in Large Language Models: Benchmarking Fact Recall Across Time
- 2025When LLM Meets Time Series: Can LLMs Perform Multi-Step Time Series Reasoning and Inference
- 2025TransientTables: Evaluating LLMs' Reasoning on Temporally Evolving Semi-structured Tables
- 2025ChronoSense: Exploring Temporal Understanding in Large Language Models with Time Intervals of Events
- 2025Toward Reasoning-Centric Time-Series Analysis