mlsanityData to 14 Oct 2026

SaulLM-7B: A pioneering Large Language Model for Law

arXiv 2403.03883 ↗First posted 6 Mar 2024cs.CLcited by 142

Not on the map. Why?

Abstract (arXiv)

In this paper, we introduce SaulLM-7B, a large language model (LLM) tailored for the legal domain. With 7 billion parameters, SaulLM-7B is the first LLM designed explicitly for legal text comprehension and generation. Leveraging the Mistral 7B architecture as its foundation, SaulLM-7B is trained on an English legal corpus of over 30 billion tokens. SaulLM-7B exhibits state-of-the-art proficiency in understanding and processing legal documents. Additionally, we present a novel instructional fine-tuning method that leverages legal datasets to further enhance SaulLM-7B's performance in legal tasks. SaulLM-7B is released under the MIT License.

Key facts. Cited by 142 papers in the map (data through 14 Oct 2026). Builds on 55 of its 76 references that are in the map. First posted on arXiv 6 Mar 2024.

Builds on

55 of 76 references.

References · 55

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  4. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  5. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  6. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,718
  7. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  8. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  9. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
  10. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
  11. 2022Scaling Instruction-Finetuned Language Modelscited by 3,024
  12. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,724
38 more
  1. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,598
  2. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,411
  3. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,218
  4. 2023Code Llama: Open Foundation Models for Codecited by 2,139
  5. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,984
  6. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,868
  7. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,786
  8. 2020Don't Stop Pretraining: Adapt Language Models to Domains and Taskscited by 1,565
  9. 2023LIMA: Less Is More for Alignmentcited by 1,100
  10. 2022Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Taskscited by 868
  11. 2023Enhancing Chat Language Models by Scaling High-quality Instructional Conversationscited by 839
  12. 2023The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Onlycited by 763
  13. 2022Galactica: A Large Language Model for Sciencecited by 744
  14. 2023DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvaturecited by 726
  15. 2021Deduplicating Training Data Makes Language Models Bettercited by 714
  16. 2024TinyLlama: An Open-Source Small Language Modelcited by 618
  17. 2021Datasets: A Community Library for Natural Language Processingcited by 530
  18. 2023How Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resourcescited by 408
  19. 2023Orca: Progressive Learning from Complex Explanation Traces of GPT-4cited by 379
  20. 2022The Stack: 3 TB of permissively licensed source codecited by 359
  21. 2019Neural Legal Judgment Prediction in Englishcited by 209
  22. 2021Lawformer: A Pre-trained Language Model for Chinese Legal Long Documentscited by 116
  23. 2022Pile of Law: Learning Responsible Data Filtering from the Law and a 256GB Open-Source Legal Datasetcited by 113
  24. 2023Testing of Detection Tools for AI-Generated Textcited by 90
  25. 2023At Which Training Stage Does Code Data Help LLMs Reasoning?cited by 82
  26. 2023SlimPajama-DC: Understanding Data Combinations for LLM Trainingcited by 72
  27. 2023Searching for Needles in a Haystack: On the Role of Incidental Bilingualism in PaLM's Translation Capabilitycited by 62
  28. 2021Swiss-Judgment-Prediction: A Multilingual Legal Judgment Prediction Benchmarkcited by 60
  29. 2023MultiLegalPile: A 689GB Multilingual Legal Corpuscited by 60
  30. 2024Adapting Large Language Models for Document-Level Machine Translationcited by 59
  31. 2024CroissantLLM: A Truly Bilingual French-English Language Modelcited by 53
  32. 2023Adapting Large Language Models to Domains via Reading Comprehensioncited by 50
  33. 2023Explaining Legal Concepts with Augmented Large Language Models (GPT-4)cited by 40
  34. 2021Adapt-and-Distill: Developing Small, Fast and Effective Pretrained Language Models for Domainscited by 36
  35. 2024Better Call GPT, Comparing Large Language Models Against Lawyerscited by 26
  36. 2023Domain-specific Continued Pretraining of Language Models for Capturing Long Context in Mental Healthcited by 25
  37. 2022LegalBench: Prototyping a Collaborative Benchmark for Legal Reasoningcited by 17
  38. 2019Unsupervised Domain Adaptation of Language Models for Reading Comprehensioncited by 15

50 most cited of 55 All 55 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 44; 2025: 50; 2026: 48.

142 papers.

Sort:

By area:

  1. 2024Continual Learning of Large Language Models: A Comprehensive Surveycited by 234
  2. 2024Large Language Model (LLM) for Telecommunications: A Comprehensive Survey on Principles, Key Techniques, and Opportunitiescited by 83
  3. 2024Retrieval-Augmented Generation for Natural Language Processing: A Surveycited by 71
  4. 2024Synthetic continued pretrainingcited by 60
  5. 2024SoK: Membership Inference Attacks on LLMs are Rushing Nowhere (and How to Fix It)cited by 59
  6. 2024Natural Language Processing for the Legal Domain: A Survey of Tasks, Datasets, Models, and Challengescited by 42
  7. 2024TelecomGPT: A Framework to Build Telecom-Specfic Large Language Modelscited by 40
  8. 2024SaulLM-54B & SaulLM-141B: Scaling Up Domain Adaptation for the Legal Domaincited by 39
  9. 2024GoldCoin: Grounding Large Language Models in Privacy Laws via Contextual Integrity Theorycited by 37
  10. 2024LawLLM: Law Large Language Model for the US Legal Systemcited by 37
  11. 2025A Survey of Scientific Large Language Models: From Data Foundations to Agent Frontierscited by 37
  12. 2025Llama-3.1-FoundationAI-SecurityLLM-Base-8B Technical Reportcited by 31
38 more
  1. 2025ECom-Bench: Can LLM Agent Resolve Real-World E-commerce Customer Support Issues?cited by 27
  2. 2025WikiBigEdit: Understanding the Limits of Lifelong Knowledge Editing in LLMscited by 26
  3. 2024Tele-LLMs: A Series of Specialized Large Language Models for Telecommunicationscited by 25
  4. 2025Demystifying Domain-adaptive Post-training for Financial LLMscited by 24
  5. 2024CMR Scaling Law: Predicting Critical Mixture Ratios for Continual Pre-training of Language Modelscited by 23
  6. 2025Primus: A Pioneering Collection of Open-Source Datasets for Cybersecurity LLM Trainingcited by 23
  7. 2025Llama-3.1-FoundationAI-SecurityLLM-8B-Instruct Technical Reportcited by 23
  8. 2024Sparsing Law: Towards Large Language Models with Greater Activation Sparsitycited by 22
  9. 2024Can Large Language Models Generate High-quality Patent Claims?cited by 19
  10. 2025FinMTEB: Finance Massive Text Embedding Benchmarkcited by 18
  11. 2024A Survey on Large Language Models from General Purpose to Medical Applications: Datasets, Methodologies, and Evaluationscited by 16
  12. 2025Memory Decoder: A Pretrained, Plug-and-Play Memory for Large Language Modelscited by 16
  13. 2024Do We Need Domain-Specific Embedding Models? An Empirical Investigationcited by 12
  14. 2025Automatic Legal Writing Evaluation of LLMscited by 12
  15. 2024Extend Model Merging from Fine-Tuned to Pre-Trained Large Language Models via Weight Disentanglementcited by 11
  16. 2025Domaino1s: Guiding LLM Reasoning for Explainable Answers in High-Stakes Domainscited by 11
  17. 2024LawInstruct: A Resource for Studying Language Model Adaptation to the Legal Domaincited by 10
  18. 2024The Factuality of Large Language Models in the Legal Domaincited by 10
  19. 2025SOFT: Selective Data Obfuscation for Protecting LLM Fine-tuning against Membership Inference Attackscited by 10
  20. 2025Large Language Models Meet Legal Artificial Intelligence: A Surveycited by 10
  21. 2025Domain Adaptation of Foundation LLMs for e-Commercecited by 9
  22. 2025When Continue Learning Meets Multimodal Large Language Model: A Surveycited by 9
  23. 2025ContractEval: Benchmarking LLMs for Clause-Level Legal Risk Identification in Commercial Contractscited by 9
  24. 2025GraphCompliance: Structure-Aware Regulatory Compliance through Policy-Context Graph Alignmentcited by 9
  25. 2025TRIDENT: Benchmarking LLM Safety in Finance, Medicine, and Lawcited by 8
  26. 2024Lawma: The Power of Specialization for Legal Annotationcited by 7
  27. 2024Revolutionizing Database Q&A with Large Language Models: Comprehensive Benchmark and Evaluationcited by 7
  28. 2025A Law Reasoning Benchmark for LLM with Tree-Organized Structures including Factum Probandum, Evidence and Experiencescited by 7
  29. 2025Middo: Model-Informed Dynamic Data Optimization for Enhanced LLM Fine-Tuning via Closed-Loop Learningcited by 7
  30. 2025Building High-Quality Datasets for Portuguese LLMs: From Common Crawl Snapshots to Industrial-Grade Corporacited by 7
  31. 2024LegiLM: A Fine-Tuned Legal Language Model for Data Compliancecited by 6
  32. 2024Opportunities and Challenges of Generative-AI in Financecited by 6
  33. 2024LAW: Legal Agentic Workflows for Custody and Fund Services Contractscited by 6
  34. 2025OntoURL: A Benchmark for Evaluating Large Language Models on Symbolic Ontological Understanding, Reasoning and Learningcited by 6
  35. 2025GuRE:Generative Query REwriter for Legal Passage Retrievalcited by 6
  36. 2025RedOne: Revealing Domain-specific LLM Post-Training in Social Networking Servicescited by 6
  37. 2025CoCoLex: Confidence-guided Copy-based Decoding for Grounded Legal Text Generationcited by 6
  38. 2025AIReg-Bench: Benchmarking Language Models That Assess AI Regulation Compliancecited by 6

50 most cited of 142 All 142 (JSON)