# SynLexLM: Scaling Legal LLMs with Synthetic Data and Curriculum Learning

arXiv: https://arxiv.org/abs/2504.18762 · HTML: https://mlsanity.com/paper/2504.18762/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in LLM synthetic data generation, one of 13 directions in Synthetic data and self-improvement for LLMs.
- Builds on 9 of its 16 references that are in the map.
- First posted on arXiv 26 Apr 2025.

## Placed in

- Synthetic data and self-improvement for LLMs / [LLM synthetic data generation](https://mlsanity.com/synth/llm-synthetic-data-generation/)

## Builds on (most cited first, 9 of 9)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [SeqGAN: Sequence Generative Adversarial Nets with Policy Gradient](https://mlsanity.com/paper/1609.05473/) (2016)
- [LexGLUE: A Benchmark Dataset for Legal Language Understanding in English](https://mlsanity.com/paper/2110.00976/) (2021)
- [CUAD: An Expert-Annotated NLP Dataset for Legal Contract Review](https://mlsanity.com/paper/2103.06268/) (2021)
- [Synthetic Data for Deep Learning](https://mlsanity.com/paper/1909.11512/) (2019)
- [EUR-Lex-Sum: A Multi- and Cross-lingual Dataset for Long-form Summarization in the Legal Domain](https://mlsanity.com/paper/2210.13448/) (2022)
- [A Dataset of German Legal Documents for Named Entity Recognition](https://mlsanity.com/paper/2003.13016/) (2020)
- [Generation of Synthetic Electronic Medical Record Text](https://mlsanity.com/paper/1812.02793/) (2018)

## Built on by (most cited first, 4 of 4)

- [LaMDAgent: An Autonomous Framework for Post-Training Pipeline Optimization via LLM Agents](https://mlsanity.com/paper/2505.21963/) (2025)
- [AQuilt: Weaving Logic and Self-Inspection into Low-Cost, High-Relevance Data Synthesis for Specialist LLMs](https://mlsanity.com/paper/2507.18584/) (2025)
- [Movable Antenna Enhanced Federated Fine-Tuning of Large Language Models via Hybrid Client Selection Optimization](https://mlsanity.com/paper/2506.00011/) (2025)
- [Hidden Threat in Synthetic Data: Covert Targeted Bias Injection through Benign Text](https://mlsanity.com/paper/2608.30619/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) are powerful but often require extensive fine-tuning and large datasets for specialized domains like law. General-purpose pre-training may not capture legal nuances, and acquiring sufficient legal data is challenging. We introduce SynLexLM, a novel approach to efficiently pre-train a legal LLM. Our method employs curriculum learning, progressing from simple to complex legal texts and queries, combined with synthetic data augmentation using models like Gemini Pro to address data scarcity. We aim to achieve improved performance on legal benchmarks (BigLaw-Bench, EUR-Lex-Sum) compared to traditional models and fine-tuned versions. Preliminary work involves generating synthetic QA pairs reflecting legal reasoning. This work aims to enhance legal document analysis and research tools, potentially democratizing access to advanced legal AI.