LexKairos: Benchmarking Legal Temporal Capabilities in LLMs
Not on the map. Why?
Abstract (arXiv)
Large language models (LLMs) have demonstrated strong performance across a wide range of legal tasks. In legal practice, time is a critical concept that governs the validity of statutes, the progression of legal cases, and the enforcement of procedural deadlines. However, legal temporal capabilities remain underexplored in existing legal AI benchmarks. To address this gap, we propose LexKairos, a comprehensive benchmark for evaluating the temporal capabilities of LLMs in the Chinese legal context across three dimensions: statutory temporal knowledge, case temporal modeling, and statute-case temporal reasoning. LexKairos comprises nine sub-tasks drawn from real-world Chinese judicial cases and statutes. We conduct systematic evaluations of eight LLMs under multiple inference settings, including vanilla, Chain-of-Thought (CoT), and thinking modes. Our results show that Gemini-3-Flash achieves the strongest overall performance, yet even the best-performing model exhibits notable limitations on tasks demanding precise time-sensitive statutory metadata recall or complex reasoning in time limits, indicating that legal temporal knowledge and reasoning remain open challenges for current LLMs. Data and code are available at https://github.com/thunlp/LexKairos.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 13 of its 29 references that are in the map. First posted on arXiv 10 Aug 2026.
Builds on
13 of 29 references.
References · 13
- 2026DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligencecited by 877
- 2019Neural Legal Judgment Prediction in Englishcited by 208
- 2018CAIL2018: A Large-Scale Legal Dataset for Judgment Predictioncited by 168
- 2023Legal Syllogism Prompting: Teaching Large Language Models for Legal Judgment Predictioncited by 37
- 2024Time Awareness in Large Language Models: Benchmarking Fact Recall Across Timecited by 12
- 2025LexTime: A Benchmark for Temporal Ordering of Legal Eventscited by 9
- 2024The CLC-UKET Dataset: Benchmarking Case Outcome Prediction for the UK Employment Tribunalcited by 5
- 2025LexChain: Modeling Legal Reasoning Chains for Chinese Tort Case Analysiscited by 3
- 2025LexGenius: An Expert-Level Benchmark for Large Language Models in Legal General Intelligencecited by 3
- 2026LegalOne: A Family of Foundation Models for Reliable Legal Reasoningcited by 3
- 2025CLaw: Benchmarking Chinese Legal Knowledge in Large Language Models - A Fine-grained Corpus and Reasoning Analysiscited by 1
- 2025LexRel: Benchmarking Legal Relation Extraction for Chinese Civil Casescited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026When Do LLMs Apply the Wrong Law? Diagnosing LLM Failures in Temporal Legal Reasoning
- 2025Evaluating Test-Time Scaling LLMs for Legal Reasoning: OpenAI o1, DeepSeek-R1, and Beyond
- 2023LawBench: Benchmarking Legal Knowledge of Large Language Models
- 2025Benchmarking Multi-Step Legal Reasoning and Analyzing Chain-of-Thought Effects in Large Language Models
- 2024LegalAgentBench: Evaluating LLM Agents in Legal Domain
- 2026Can LLMs Time Travel? Enhancing Temporal Consistency in Legal Agentic Search through Reinforcement Learning
- 2023LegalBench: A Collaboratively Built Benchmark for Measuring Legal Reasoning in Large Language Models
- 2025Better Call CLAUSE: A Discrepancy Benchmark for Auditing LLMs Legal Reasoning Capabilities