# The Massive Legal Embedding Benchmark (MLEB)

arXiv: https://arxiv.org/abs/2510.19365 · HTML: https://mlsanity.com/paper/2510.19365/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 19 references that are in the map.
- First posted on arXiv 22 Oct 2025.

## Builds on (most cited first, 7 of 7)

- [LegalBench: A Collaboratively Built Benchmark for Measuring Legal Reasoning in Large Language Models](https://mlsanity.com/paper/2308.11462/) (2023)
- [CUAD: An Expert-Annotated NLP Dataset for Legal Contract Review](https://mlsanity.com/paper/2103.06268/) (2021)
- [ContractNLI: A Dataset for Document-level Natural Language Inference for Contracts](https://mlsanity.com/paper/2110.01799/) (2021)
- [LegalBench-RAG: A Benchmark for Retrieval-Augmented Generation in the Legal Domain](https://mlsanity.com/paper/2408.10343/) (2024)
- [A Reasoning-Focused Legal Retrieval Benchmark](https://mlsanity.com/paper/2505.03970/) (2025)
- [Plain English Summarization of Contracts](https://mlsanity.com/paper/1906.00424/) (2019)
- [Inscriptis -- A Python-based HTML to text conversion library optimized for knowledge extraction from the Web](https://mlsanity.com/paper/2108.01454/) (2021)

## Built on by (most cited first, 6 of 6)

- [Mecellem Models: Turkish Models Trained from Scratch and Continually Pre-trained for the Legal Domain](https://mlsanity.com/paper/2601.16018/) (2026)
- [CanLegalRAGBench: Evaluating Retrieval-Augmented Generation on Canadian Case Law](https://mlsanity.com/paper/2605.30497/) (2026)
- [Legal RAG Bench: an end-to-end benchmark for legal RAG](https://mlsanity.com/paper/2603.01710/) (2026)
- [GraphRAG for Engineering Diagrams: ChatP&ID Enables LLM Interaction with P&IDs](https://mlsanity.com/paper/2603.22528/) (2026)
- [GreekBarRetrieval: A Benchmark for Greek Statutory Retrieval](https://mlsanity.com/paper/2608.18752/) (2026)
- [UK-PRBENCH: A Paragraph-Level Precedent Retrieval Benchmark for United Kingdom Case Law](https://mlsanity.com/paper/2609.24613/) (2026)

## Abstract (arXiv)

We present the Massive Legal Embedding Benchmark (MLEB), the largest, most diverse, and most comprehensive open-source benchmark for legal information retrieval to date. MLEB consists of ten expert-annotated datasets spanning multiple jurisdictions (the US, UK, EU, Australia, Ireland, and Singapore), document types (cases, legislation, regulatory guidance, contracts, and literature), and task types (search, zero-shot classification, and question answering). Seven of the datasets in MLEB were newly constructed in order to fill domain and jurisdictional gaps in the open-source legal information retrieval landscape. We document our methodology in building MLEB and creating the new constituent datasets, and release our code, results, and data openly to assist with reproducible evaluations.