Temporal Concept Drift in Legal Judgment Prediction: Neural Baselines Across Three Epochs of Ukrainian Court Decisions
Not on the map. Why?
Abstract (arXiv)
Legal NLP benchmarks evaluate models on randomly split data, implicitly assuming that legal language is stationary. We test this assumption by fine-tuning four transformer encoders -- XLM-RoBERTa (base and large) and their legal-domain variants -- on Ukrainian court decisions from three temporal epochs defined by geopolitical disruptions: pre-war (2008-2013), hybrid war (2014-2021), and full-scale invasion (2022-2026). Each model is trained on one epoch and evaluated on all three, producing a 3x3 cross-temporal generalization matrix. Four findings emerge. (1) Forward degradation is severe: models trained on pre-war data lose up to 27.2 percentage points of macro-F1 when applied to full-scale invasion era decisions. (2) The degradation is asymmetric: backward transfer (full-scale to pre-war) is substantially more robust than forward transfer, consistent with the hypothesis that legal language is additive. (3) Legal-domain pretraining (Legal-XLM-R) does not improve absolute performance but reduces forward degradation magnitude and asymmetry. (4) Chronological continual learning eliminates catastrophic forgetting for general XLM-R: pre-war knowledge is fully retained (+1.8 to +6.2 pp) while full-scale performance gains +16.5 to +19.0 pp; reverse-chronological training causes severe forgetting. Cross-jurisdictional pretraining on Swiss Judgment Prediction data improves absolute performance but does not reduce temporal degradation magnitude, confirming that temporal drift is an intrinsic property of legal language evolution. The dataset (428K decisions across three epochs) is publicly available as a LEXTREME contribution.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 18 of its 25 references that are in the map. First posted on arXiv 23 May 2026.
Builds on
18 of 25 references.
References · 18
- 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
- 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
- 2019Similarity of Neural Network Representations Revisitedcited by 1,755
- 2021LexGLUE: A Benchmark Dataset for Legal Language Understanding in Englishcited by 310
- 2021Time-Aware Language Models as Temporal Knowledge Basescited by 296
- 2021Mind the Gap: Assessing Temporal Generalization in Neural Language Modelscited by 230
- 2021Time Waits for No One! Analysis and Challenges of Temporal Misalignmentcited by 91
- 2023LEXTREME: A Multi-Lingual and Multi-Task Benchmark for the Legal Domaincited by 68
- 2021Swiss-Judgment-Prediction: A Multilingual Legal Judgment Prediction Benchmarkcited by 60
- 2023MultiLegalPile: A 689GB Multilingual Legal Corpuscited by 60
- 2023LeXFiles and LegalLAMA: Facilitating English Multinational Legal Language Model Developmentcited by 54
- 2022FairLex: A Multilingual Benchmark for Evaluating Fairness in Legal Text Processingcited by 49
6 more
- 2024SaulLM-54B & SaulLM-141B: Scaling Up Domain Adaptation for the Legal Domaincited by 39
- 2022An Empirical Study on Cross-X Transfer for Legal Judgment Predictioncited by 16
- 2024PILOT: Legal Case Outcome Prediction with Case Lawcited by 6
- 2026Tokenizer Fertility and Zero-Shot Performance of Foundation Models on Ukrainian Legal Text: A Comparative Studycited by 4
- 2026Temporal Decay of Co-Citation Predictability: A 20-Year Statute Retrieval Benchmark from 396M Ukrainian Court Citationscited by 2
- 2026LEMUR: A Corpus for Robust Fine-Tuning of Multilingual Law Embedding Models for Retrievalcited by 1
Built on by
Not cited yet.