mlsanityData to 7 Oct 2026

Understand Legal Documents with Contextualized Large Language Models

arXiv 2303.12135 ↗First posted 21 Mar 2023cs.CLcited by 5

Not on the map. Why?

Abstract (arXiv)

The growth of pending legal cases in populous countries, such as India, has become a major issue. Developing effective techniques to process and understand legal documents is extremely useful in resolving this problem. In this paper, we present our systems for SemEval-2023 Task 6: understanding legal texts (Modi et al., 2023). Specifically, we first develop the Legal-BERT-HSLN model that considers the comprehensive context information in both intra- and inter-sentence levels to predict rhetorical roles (subtask A) and then train a Legal-LUKE model, which is legal-contextualized and entity-aware, to recognize legal entities (subtask B). Our evaluations demonstrate that our designed models are more accurate than baselines, e.g., with an up to 15.0% better F1 score in subtask B. We achieved notable performance in the task leaderboard, e.g., 0.834 micro F1 score, and ranked No.5 out of 27 teams in subtask A.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 23 of its 41 references that are in the map. First posted on arXiv 21 Mar 2023.

Builds on

23 of 41 references.

References · 23

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  3. 2019SpanBERT: Improving Pre-training by Representing and Predicting Spanscited by 832
  4. 2019How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddingscited by 770
  5. 2020Pre-trained Models for Natural Language Processing: A Surveycited by 504
  6. 2020LUKE: Deep Contextualized Entity Representations with Entity-aware Self-attentioncited by 326
  7. 2018A Survey on Deep Learning for Named Entity Recognitioncited by 281
  8. 2015A Critical Review of Recurrent Neural Networks for Sequence Learningcited by 223
  9. 2019Pretrained Language Models for Sequential Sentence Classificationcited by 53
  10. 2019Identification of Rhetorical Roles of Sentences in Indian Legal Judgmentscited by 39
  11. 2021Semantic Segmentation of Legal Documents via Rhetorical Rolescited by 26
  12. 2022Named Entity Recognition in Indian court judgmentscited by 26
11 more
  1. 2021NRTSI: Non-Recurrent Time Series Imputationcited by 25
  2. 2022Corpus for Automatic Structuring of Legal Documentscited by 25
  3. 2018Hierarchical Neural Networks for Sequential Sentence Classification in Medical Scientific Abstractscited by 22
  4. 2021On Sampling Top-K Recommendation Evaluationcited by 20
  5. 2021mLUKE: The Power of Entity Representations in Multilingual Pretrained Language Modelscited by 19
  6. 2021Cross-Domain Multi-Task Learning for Sequential Sentence Classification in Research Paperscited by 11
  7. 2021On Estimating Recommendation Evaluation Metrics under Samplingcited by 8
  8. 2016End-to-End Subtitle Detection and Recognition for Videos in East Asian Languages via CNN Ensemble with Near-Human-Level Performancecited by 4
  9. 2022Transparent Single-Cell Set Classification with Kernel Mean Embeddingscited by 3
  10. 2022Distribution-based Sketching of Single-Cell Samplescited by 2
  11. 2020Large-scale Real-time Personalized Similar Product Recommendationscited by 1

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 4.

5 papers.

  1. 2024Large Language Models for Forecasting and Anomaly Detection: A Systematic Literature Reviewcited by 50
  2. 2024TaskCLIP: Extend Large Vision-Language Model for Task Oriented Object Detectioncited by 21
  3. 2023CloudEval-YAML: A Practical Benchmark for Cloud Configuration Generationcited by 8
  4. 2024Comprehensive evaluation of Mal-API-2019 dataset by machine learning in malware detectioncited by 8
  5. 2024Can Contrastive Learning Refine Embeddingscited by 5