SynLexLM: Scaling Legal LLMs with Synthetic Data and Curriculum Learning
On the map: Synthetic data and self-improvement for LLMs → LLM synthetic data generation
Abstract (arXiv)
Large Language Models (LLMs) are powerful but often require extensive fine-tuning and large datasets for specialized domains like law. General-purpose pre-training may not capture legal nuances, and acquiring sufficient legal data is challenging. We introduce SynLexLM, a novel approach to efficiently pre-train a legal LLM. Our method employs curriculum learning, progressing from simple to complex legal texts and queries, combined with synthetic data augmentation using models like Gemini Pro to address data scarcity. We aim to achieve improved performance on legal benchmarks (BigLaw-Bench, EUR-Lex-Sum) compared to traditional models and fine-tuned versions. Preliminary work involves generating synthetic QA pairs reflecting legal reasoning. This work aims to enhance legal document analysis and research tools, potentially democratizing access to advanced legal AI.
Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in LLM synthetic data generation, one of 13 directions in Synthetic data and self-improvement for LLMs. Builds on 9 of its 16 references that are in the map. First posted on arXiv 26 Apr 2025.
Builds on
9 of 16 references.
References · 9
- 2017Attention Is All You Needcited by 60,361
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
- 2016SeqGAN: Sequence Generative Adversarial Nets with Policy Gradientcited by 376
- 2021LexGLUE: A Benchmark Dataset for Legal Language Understanding in Englishcited by 310
- 2021CUAD: An Expert-Annotated NLP Dataset for Legal Contract Reviewcited by 236
- 2019Synthetic Data for Deep Learningcited by 113
- 2022EUR-Lex-Sum: A Multi- and Cross-lingual Dataset for Long-form Summarization in the Legal Domaincited by 40
- 2020A Dataset of German Legal Documents for Named Entity Recognitioncited by 24
- 2018Generation of Synthetic Electronic Medical Record Textcited by 14
Built on by
Citing papers per year: 2025: 3; 2026: 1.
4 papers.
- 2025LaMDAgent: An Autonomous Framework for Post-Training Pipeline Optimization via LLM Agentscited by 6
- 2025AQuilt: Weaving Logic and Self-Inspection into Low-Cost, High-Relevance Data Synthesis for Specialist LLMscited by 4
- 2025Movable Antenna Enhanced Federated Fine-Tuning of Large Language Models via Hybrid Client Selection Optimizationcited by 1
- 2026Hidden Threat in Synthetic Data: Covert Targeted Bias Injection through Benign Textcited by 0
- 2026Hidden Threat in Synthetic Data: Covert Targeted Bias Injection through Benign Text31 Aug 2026
- 2025AQuilt: Weaving Logic and Self-Inspection into Low-Cost, High-Relevance Data Synthesis for Specialist LLMs24 Jul 2025
- 2025LaMDAgent: An Autonomous Framework for Post-Training Pipeline Optimization via LLM Agents28 May 2025
- 2025Movable Antenna Enhanced Federated Fine-Tuning of Large Language Models via Hybrid Client Selection Optimization16 May 2025
Similar, not cited
Close in wording, no citation either way
- 2024Optimizing Numerical Estimation and Operational Efficiency in the Legal Domain through Large Language Models
- 2024Knowledge-Infused Legal Wisdom: Navigating LLM Consultation through the Lens of Diagnostics and Positive-Unlabeled Reinforcement Learning
- 2025LawGPT: Knowledge-Guided Data Generation and Its Application to Legal LLM
- 2023Large Language Models as Tax Attorneys: A Case Study in Legal Capabilities Emergence
- 2023DISC-LawLLM: Fine-tuning Large Language Models for Intelligent Legal Services
- 2026Domain-Adaptation through Synthetic Data: Fine-Tuning Large Language Models for German Law
- 2026Evaluation of Large Language Models in Legal Applications: Challenges, Methods, and Future Directions
- 2026BenGER: Benchmarking LLM Systems on Subsumption-Based Legal Reasoning in German Law