mlsanityData to 14 Oct 2026

BERT Goes to Law School: Quantifying the Competitive Advantage of Access to Large Legal Corpora in Contract Understanding

arXiv 1911.00473 ↗First posted 1 Nov 2019cs.CL cs.AI cs.LGcited by 16

Not on the map. Why?

Abstract (arXiv)

Fine-tuning language models, such as BERT, on domain specific corpora has proven to be valuable in domains like scientific papers and biomedical text. In this paper, we show that fine-tuning BERT on legal documents similarly provides valuable improvements on NLP tasks in the legal domain. Demonstrating this outcome is significant for analyzing commercial agreements, because obtaining large legal corpora is challenging due to their confidential nature. As such, we show that having access to large legal corpora is a competitive advantage for commercial applications, and academic research on analyzing contracts.

Key facts. Cited by 16 papers in the map (data through 14 Oct 2026). First posted on arXiv 1 Nov 2019.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 6; 2022: 5; 2023: 2; 2024: 2; 2025: 1.

16 papers.

Sort:

  1. 2021When Does Pretraining Help? Assessing Self-Supervised Learning for Law and the CaseHOLD Datasetcited by 201
  2. 2021Lawformer: A Pre-trained Language Model for Chinese Legal Long Documentscited by 116
  3. 2021Factoring Statutory Reasoning as Language Understanding Challengescited by 20
  4. 2021JuriBERT: A Masked-Language Model Adaptation for French Legal Textcited by 14
  5. 2022Privacy-Preserving Models for Legal Natural Language Processingcited by 10
  6. 2021Customizing Contextualized Language Models forLegal Document Reviewscited by 8
  7. 2022A Comparison of SVM against Pre-trained Language Models (PLMs) for Text Classification Taskscited by 7
  8. 2021The Law of Large Documents: Understanding the Structure of Legal Contracts Using Visual Cuescited by 4
  9. 2022Better Transcription of UK Supreme Court Hearingscited by 3
  10. 2022Enhancing Legal Argument Mining with Domain Pre-training and Neural Networkscited by 2
  11. 2022DeeperDive: The Unreasonable Effectiveness of Weak Supervision in Document Understanding A Case Study in Collaboration with UiPath Inccited by 2
  12. 2023Classification of US Supreme Court Cases using BERT-Based Techniquescited by 2
4 more
  1. 2024Annotation and Classification of Relevant Clauses in Terms-and-Conditions Contractscited by 1
  2. 2024A Small Claims Court for the NLP: Judging Legal Text Classification Strategies With Small Datasetscited by 1
  3. 2023Contrast Is All You Needcited by 0
  4. 2025Health Insurance Coverage Rule Interpretation Corpus: Law, Policy, and Medical Guidance for Health Insurance Coverage Understandingcited by 0