# MASLegalBench: Benchmarking Multi-Agent Systems in Deductive Legal Reasoning

arXiv: https://arxiv.org/abs/2509.24922 · HTML: https://mlsanity.com/paper/2509.24922/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning.
- Builds on 16 of its 30 references that are in the map.
- First posted on arXiv 29 Sep 2025.

## Placed in

- LLM reasoning / [Reasoning for agents, planning and code](https://mlsanity.com/lr/reasoning-for-agents-planning-and-code/)
- LLM agents and tool use / [Social simulation and multi-agent debate](https://mlsanity.com/agents/social-simulation-and-multi-agent-debate/)

## Builds on (most cited first, 16 of 16)

- [Towards Reasoning in Large Language Models: A Survey](https://mlsanity.com/paper/2212.10403/) (2022)
- [Agent Hospital: A Simulacrum of Hospital with Evolvable Medical Agents](https://mlsanity.com/paper/2405.02957/) (2024)
- [Chatlaw: A Multi-Agent Legal Assistant based on a Role-Aligned Mixture-of-Experts Architecture](https://mlsanity.com/paper/2306.16092/) (2023)
- [LawBench: Benchmarking Legal Knowledge of Large Language Models](https://mlsanity.com/paper/2309.16289/) (2023)
- [A Survey of Frontiers in LLM Reasoning: Inference Scaling, Learning to Reason, and Agentic Systems](https://mlsanity.com/paper/2504.09037/) (2025)
- [LegalAgentBench: Evaluating LLM Agents in Legal Domain](https://mlsanity.com/paper/2412.17259/) (2024)
- [TwinMarket: A Scalable Behavioral and Social Simulation for Financial Markets](https://mlsanity.com/paper/2502.01506/) (2025)
- [SDPO: Segment-Level Direct Preference Optimization for Social Agents](https://mlsanity.com/paper/2501.01821/) (2025)
- [MAS-ZERO: Designing Multi-Agent Systems with Zero Supervision](https://mlsanity.com/paper/2505.14996/) (2025)
- [Chain of Logic: Rule-Based Reasoning with Large Language Models](https://mlsanity.com/paper/2402.10400/) (2024)
- [Can Large Language Models Grasp Legal Theories? Enhance Legal Reasoning with Insights from Multi-Agent Collaboration](https://mlsanity.com/paper/2410.02507/) (2024)
- [ACORD: An Expert-Annotated Retrieval Dataset for Legal Contract Drafting](https://mlsanity.com/paper/2501.06582/) (2025)
- [AI Agents and the Law](https://mlsanity.com/paper/2508.08544/) (2025)
- [aiXiv: A Next-Generation Open Access Ecosystem for Scientific Discovery Generated by AI Scientists](https://mlsanity.com/paper/2508.15126/) (2025)
- [Computing and Exploiting Document Structure to Improve Unsupervised Extractive Summarization of Legal Case Decisions](https://mlsanity.com/paper/2211.03229/) (2022)
- [Multi Agent based Medical Assistant for Edge Devices](https://mlsanity.com/paper/2503.05397/) (2025)

## Built on by (most cited first, 3 of 3)

- [LLM Agents in Law: Taxonomy, Applications, and Challenges](https://mlsanity.com/paper/2601.06216/) (2026)
- [LawThinker: A Deep Research Legal Agent in Dynamic Environments](https://mlsanity.com/paper/2602.12056/) (2026)
- [LabourCrew: A Multi-Agent RAG Framework for Trustworthy Adversarial Deliberation and Statutory Reasoning over Labour Law](https://mlsanity.com/paper/2609.27814/) (2026)

## Abstract (arXiv)

Multi-agent systems (MAS), leveraging the remarkable capabilities of Large Language Models (LLMs), show great potential in addressing complex tasks. In this context, integrating MAS with legal tasks is a crucial step. While previous studies have developed legal benchmarks for LLM agents, none are specifically designed to consider the unique advantages of MAS, such as task decomposition, agent specialization, and flexible training. In fact, the lack of evaluation methods limits the potential of MAS in the legal domain. To address this gap, we propose MASLegalBench, a legal benchmark tailored for MAS and designed with a deductive reasoning approach. Our benchmark uses GDPR as the application scenario, encompassing extensive background knowledge and covering complex reasoning processes that effectively reflect the intricacies of real-world legal situations. Furthermore, we manually design various role-based MAS and conduct extensive experiments using different state-of-the-art LLMs. Our results highlight the strengths, limitations, and potential areas for improvement of existing models and MAS architectures.