mlsanityData to 7 Oct 2026

Improving Legal Judgement Prediction in Romanian with Long Text Encoders

arXiv 2402.19170 ↗First posted 29 Feb 2024cs.CL cs.AIcited by 3

Not on the map. Why?

Abstract (arXiv)

In recent years,the entire field of Natural Language Processing (NLP) has enjoyed amazing novel results achieving almost human-like performance on a variety of tasks. Legal NLP domain has also been part of this process, as it has seen an impressive growth. However, general-purpose models are not readily applicable for legal domain. Due to the nature of the domain (e.g. specialized vocabulary, long documents) specific models and methods are often needed for Legal NLP. In this work we investigate both specialized and general models for predicting the final ruling of a legal case, task known as Legal Judgment Prediction (LJP). We particularly focus on methods to extend to sequence length of Transformer-based models to better understand the long documents present in legal corpora. Extensive experiments on 4 LJP datasets in Romanian, originating from 2 sources with significantly different sizes and document lengths, show that specialized models and handling long texts are critical for a good performance.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 36 of its 50 references that are in the map. First posted on arXiv 29 Feb 2024.

Builds on

36 of 50 references.

References · 36

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  4. 2020Language Models are Few-Shot Learnerscited by 32,199
  5. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  6. 2022Training language models to follow instructions with human feedbackcited by 15,712
  7. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  8. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  9. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  10. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  11. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
  12. 2020Longformer: The Long-Document Transformercited by 3,362
24 more
  1. 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,935
  2. 2020Reformer: The Efficient Transformercited by 1,658
  3. 2019Generating Long Sequences with Sparse Transformerscited by 1,523
  4. 2023BloombergGPT: A Large Language Model for Financecited by 923
  5. 2020Efficient Content-Based Sparse Attention with Routing Transformerscited by 469
  6. 2020Med-BERT: pre-trained contextualized embeddings on large-scale structured electronic health records for disease predictioncited by 368
  7. 2020ChemBERTa: Large-Scale Self-Supervised Pretraining for Molecular Property Predictioncited by 332
  8. 2022CogView2: Faster and Better Text-to-Image Generation via Hierarchical Transformerscited by 305
  9. 2021LexGLUE: A Benchmark Dataset for Legal Language Understanding in Englishcited by 302
  10. 2021CUAD: An Expert-Annotated NLP Dataset for Legal Contract Reviewcited by 232
  11. 2021Keeping Your Eye on the Ball: Trajectory Attention in Video Transformerscited by 224
  12. 2019Neural Legal Judgment Prediction in Englishcited by 208
  13. 2023Okapi: Instruction-tuned Large Language Models in Multiple Languages with Reinforcement Learning from Human Feedbackcited by 189
  14. 2020FinBERT: A Pretrained Language Model for Financial Communicationscited by 161
  15. 2022ChemBERTa-2: Towards Chemical Foundation Modelscited by 123
  16. 2019Large-Scale Multi-Label Text Classification on EU Legislationcited by 104
  17. 2021Paragraph-level Rationale Extraction through Regularization: A case study on European Court of Human Rights Casescited by 77
  18. 2022Efficient Long-Text Understanding with Short-Text Modelscited by 71
  19. 2023LEXTREME: A Multi-Lingual and Multi-Task Benchmark for the Legal Domaincited by 68
  20. 2020An Empirical Study on Large-Scale Multi-Label Text Classification Including Few and Zero-Shot Labelscited by 44
  21. 2022KPI-BERT: A Joint Named Entity Recognition and Relation Extraction Model for Financial Reportscited by 19
  22. 2022An Empirical Study on Cross-X Transfer for Legal Judgment Predictioncited by 16
  23. 2021Dependency Learning for Legal Judgment Prediction with a Unified Text-to-Text Transformercited by 13
  24. 2022BudgetLongformer: Can we Cheaply Pretrain a SotA Legal Language Model From Scratch?cited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 1; 2026: 1.

3 papers.

  1. 2025LegalReasoner: Step-wised Verification-Correction for Legal Judgment Reasoningcited by 9
  2. 2024GRAF: Graph Retrieval Augmented by Facts for Romanian Legal Multi-Choice Question Answeringcited by 4
  3. 2026HKJudge: A Legal Discourse-Annotated Corpus for Interpreting What Courts Find, How They Reason, and What They Rulecited by 0