mlsanityData to 7 Oct 2026

Classifying long legal documents using short random chunks

arXiv 2512.24997 ↗First posted 31 Dec 2025cs.CL cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Classifying legal documents is a challenge, besides their specialized vocabulary, sometimes they can be very long. This means that feeding full documents to a Transformers-based models for classification might be impossible, expensive or slow. Thus, we present a legal document classifier based on DeBERTa V3 and a LSTM, that uses as input a collection of 48 randomly-selected short chunks (max 128 tokens). Besides, we present its deployment pipeline using Temporal, a durable execution solution, which allow us to have a reliable and robust processing workflow. The best model had a weighted F-score of 0.898, while the pipeline running on CPU had a processing median time of 498 seconds per 100 files.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 16 of its 29 references that are in the map. First posted on arXiv 31 Dec 2025.

Builds on

16 of 29 references.

References · 16

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  4. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  5. 2021DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharingcited by 1,219
  6. 2014Distributed Representations of Sentences and Documentscited by 677
  7. 2019Lookahead Optimizer: k steps forward, 1 step backcited by 333
  8. 2018Explainable Prediction of Medical Codes from Clinical Textcited by 231
  9. 2020How Does NLP Benefit Legal System: A Summary of Legal Artificial Intelligencecited by 128
  10. 2021Lawformer: A Pre-trained Language Model for Chinese Legal Long Documentscited by 115
  11. 2019Large-Scale Multi-Label Text Classification on EU Legislationcited by 104
  12. 2019Hierarchical Transformers for Long Document Classificationcited by 103
4 more
  1. 2022Efficient Classification of Long Documents Using Transformerscited by 30
  2. 2022Processing Long Legal Documents with Pre-trained Transformers: Modding LegalBERT and Longformercited by 20
  3. 2021Comparative Study of Long Document Classificationcited by 12
  4. 2019Long-length Legal Document Classificationcited by 10

Built on by

Not cited yet.