mlsanityData to 7 Oct 2026

ChronosLex: Time-aware Incremental Training for Temporal Generalization of Legal Classification Tasks

arXiv 2405.14211 ↗First posted 23 May 2024cs.CLcited by 11

Not on the map. Why?

Abstract (arXiv)

This study investigates the challenges posed by the dynamic nature of legal multi-label text classification tasks, where legal concepts evolve over time. Existing models often overlook the temporal dimension in their training process, leading to suboptimal performance of those models over time, as they treat training data as a single homogeneous block. To address this, we introduce ChronosLex, an incremental training paradigm that trains models on chronological splits, preserving the temporal order of the data. However, this incremental approach raises concerns about overfitting to recent data, prompting an assessment of mitigation strategies using continual learning and temporal invariant methods. Our experimental results over six legal multi-label text classification datasets reveal that continual learning methods prove effective in preventing overfitting thereby enhancing temporal generalizability, while temporal invariant methods struggle to capture these dynamics of temporal shifts.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 63 of its 92 references that are in the map. First posted on arXiv 23 May 2024.

Builds on

63 of 92 references.

References · 63

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  2. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  3. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  4. 2015Domain-Adversarial Training of Neural Networkscited by 3,023
  5. 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,313
  6. 2017Gradient Episodic Memory for Continual Learningcited by 1,871
  7. 2019Invariant Risk Minimizationcited by 1,832
  8. 2016Deep CORAL: Correlation Alignment for Deep Domain Adaptationcited by 1,194
  9. 2018Experience Replay for Continual Learningcited by 1,074
  10. 2017Lifelong Learning with Dynamically Expandable Networkscited by 689
  11. 2019Gradient based sample selection for online continual learningcited by 568
  12. 2020AdapterHub: A Framework for Adapting Transformerscited by 441
38 more
  1. 2021LexGLUE: A Benchmark Dataset for Legal Language Understanding in Englishcited by 302
  2. 2021Time-Aware Language Models as Temporal Knowledge Basescited by 290
  3. 2021CUAD: An Expert-Annotated NLP Dataset for Legal Contract Reviewcited by 232
  4. 2021Mind the Gap: Assessing Temporal Generalization in Neural Language Modelscited by 226
  5. 2019Neural Legal Judgment Prediction in Englishcited by 208
  6. 2021When Does Pretraining Help? Assessing Self-Supervised Learning for Law and the CaseHOLD Datasetcited by 199
  7. 2019Episodic Memory in Lifelong Language Learningcited by 188
  8. 2021Towards Continual Knowledge Learning of Language Modelscited by 172
  9. 2020Recall and Learn: Fine-tuning Deep Pretrained Language Models with Less Forgettingcited by 169
  10. 2019LAMOL: LAnguage MOdeling for Lifelong Language Learningcited by 156
  11. 2020Continual Lifelong Learning in Natural Language Processing: A Surveycited by 152
  12. 2021DEMix Layers: Disentangling Domains for Modular Language Modelingcited by 130
  13. 2022TimeLMs: Diachronic Language Models from Twittercited by 129
  14. 2022StreamingQA: A Benchmark for Adaptation to New Knowledge over Time in Question Answering Modelscited by 123
  15. 2022TemporalWiki: A Lifelong Benchmark for Training and Evaluating Ever-Evolving Language Modelscited by 117
  16. 2021Lifelong Pretraining: Continually Adapting Language Models to Emerging Corporacited by 114
  17. 2021Time Waits for No One! Analysis and Challenges of Temporal Misalignmentcited by 91
  18. 2021MultiEURLEX -- A multi-lingual and multi-label legal document classification dataset for zero-shot cross-lingual transfercited by 89
  19. 2021Continual Learning for Text Classification with Information Disentanglement Based Regularizationcited by 86
  20. 2022Wild-Time: A Benchmark of in-the-Wild Distribution Shift over Timecited by 85
  21. 2022Continual Learning of Natural Language Processing Tasks: A Surveycited by 83
  22. 2021ILDC for CJPE: Indian Legal Documents Corpus for Court Judgment Prediction and Explanationcited by 78
  23. 2021Paragraph-level Rationale Extraction through Regularization: A case study on European Court of Human Rights Casescited by 77
  24. 2018CLAUDETTE: an Automated Detector of Potentially Unfair Clauses in Online Terms of Servicecited by 72
  25. 2019Sentence Embedding Alignment for Lifelong Relation Extractioncited by 70
  26. 2020We Need to Talk About Random Splitscited by 70
  27. 2022ELLE: Efficient Lifelong Pre-training for Emerging Datacited by 70
  28. 2021Swiss-Judgment-Prediction: A Multilingual Legal Judgment Prediction Benchmarkcited by 60
  29. 2022Entity Cloze By Date: What LMs Know About Unseen Entitiescited by 55
  30. 2021Temporal Adaptation of BERT and Performance on Downstream Document Classification: Insights from Social Mediacited by 51
  31. 2021Time Masking for Temporal Language Modelscited by 46
  32. 2021Temporal Effects on Pre-trained Models for Language Processing Taskscited by 46
  33. 2021Adapting BERT for Continual Learning of a Sequence of Aspect Sentiment Classification Taskscited by 46
  34. 2020An Empirical Study on Large-Scale Multi-Label Text Classification Including Few and Zero-Shot Labelscited by 44
  35. 2017Predicting the Law Area and Decisions of French Supreme Court Casescited by 30
  36. 2021Dynamic Language Models for Continuously Evolving Contentcited by 27
  37. 2022Improving Time Sensitivity for Question Answering over Temporal Knowledge Graphscited by 26
  38. 2022Deconfounding Legal Judgment Prediction for European Court of Human Rights Cases Towards Better Alignment with Expertscited by 24

50 most cited of 63 All 63 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 6; 2026: 3.

11 papers.

  1. 2025It's High Time: A Survey of Temporal Question Answeringcited by 12
  2. 2025Examining and Adapting Time for Multilingual Classification via Mixture of Temporal Expertscited by 9
  3. 2024LexSumm and LexT5: Benchmarking and Modeling Legal Summarization Tasks in Englishcited by 7
  4. 2025CoCoLex: Confidence-guided Copy-based Decoding for Grounded Legal Text Generationcited by 6
  5. 2026Taxonomy of the Retrieval System Framework: Pitfalls and Paradigmscited by 4
  6. 2025LexGenie: Automated Generation of Structured Reports for European Court of Human Rights Case Lawcited by 2
  7. 2025RELexED: Retrieval-Enhanced Legal Summarization with Exemplar Diversitycited by 1
  8. 2025AIMSCheck: Leveraging LLMs for AI-Assisted Review of Modern Slavery Statements Across Jurisdictionscited by 1
  9. 2026Knowledge-driven Augmentation and Retrieval for Integrative Temporal Adaptationcited by 1
  10. 2026Can LLMs Time Travel? Enhancing Temporal Consistency in Legal Agentic Search through Reinforcement Learningcited by 1
  11. 2024Towards Supporting Legal Argumentation with NLP: Is More Data Really All You Need?cited by 0