mlsanityData to 14 Oct 2026

AraLegal-BERT: A pretrained language model for Arabic Legal text

arXiv 2210.08284 ↗First posted 15 Oct 2022cs.CLcited by 7

Not on the map. Why?

Abstract (arXiv)

The effectiveness of the BERT model on multiple linguistic tasks has been well documented. On the other hand, its potentials for narrow and specific domains such as Legal, have not been fully explored. In this paper, we examine how BERT can be used in the Arabic legal domain and try customizing this language model for several downstream tasks using several different domain-relevant training and testing datasets to train BERT from scratch. We introduce the AraLegal-BERT, a bidirectional encoder Transformer-based model that have been thoroughly tested and carefully optimized with the goal to amplify the impact of NLP-driven solution concerning jurisprudence, legal documents, and legal practice. We fine-tuned AraLegal-BERT and evaluated it against three BERT variations for Arabic language in three natural languages understanding (NLU) tasks. The results show that the base version of AraLegal-BERT achieve better accuracy than the general and original BERT over the Legal text.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 15 of its 23 references that are in the map. First posted on arXiv 15 Oct 2022.

Builds on

15 of 23 references.

References · 15

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,947
  5. 2019SciBERT: A Pretrained Language Model for Scientific Textcited by 1,414
  6. 2019Publicly Available Clinical BERT Embeddingscited by 1,002
  7. 2020A Primer in BERTology: What we know about how BERT workscited by 815
  8. 2019How to Fine-Tune BERT for Text Classification?cited by 411
  9. 2020AraBERT: Transformer-based Model for Arabic Language Understandingcited by 362
  10. 2017Recent Trends in Deep Learning Based Natural Language Processingcited by 263
  11. 2020ARBERT & MARBERT: Deep Bidirectional Transformers for Arabiccited by 166
  12. 2020LEGAL-BERT: The Muppets straight out of Law Schoolcited by 111
3 more
  1. 2021A Comprehensive Survey on Community Detection with Deep Learningcited by 63
  2. 2021Pre-Training BERT on Arabic Tweets: Practical Considerationscited by 39
  3. 2022Enhancing Legal Argument Mining with Domain Pre-training and Neural Networkscited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 4; 2025: 0; 2026: 1.

7 papers.

  1. 2023LEXTREME: A Multi-Lingual and Multi-Task Benchmark for the Legal Domaincited by 68
  2. 2024Natural Language Processing for the Legal Domain: A Survey of Tasks, Datasets, Models, and Challengescited by 42
  3. 2024A Survey of Large Language Models for Arabic Language and its Dialectscited by 13
  4. 2024LexSumm and LexT5: Benchmarking and Modeling Legal Summarization Tasks in Englishcited by 8
  5. 2023Prediction of Arabic Legal Rulings using Large Language Modelscited by 6
  6. 2024EgyBERT: A Large Language Model Pretrained on Egyptian Dialect Corporacited by 5
  7. 2026Parameter Efficient Fine Tuning Llama 3.1 for Answering Arabic Legal Questions: A Case Study on Jordanian Lawscited by 0