# Towards Trustworthy Legal AI through LLM Agents and Formal Reasoning

arXiv: https://arxiv.org/abs/2511.21033 · HTML: https://mlsanity.com/paper/2511.21033/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning.
- Builds on 19 of its 32 references that are in the map.
- First posted on arXiv 26 Nov 2025.

## Placed in

- LLM reasoning / [Reasoning for agents, planning and code](https://mlsanity.com/lr/reasoning-for-agents-planning-and-code/)
- LLM agents and tool use / [Social simulation and multi-agent debate](https://mlsanity.com/agents/social-simulation-and-multi-agent-debate/)

## Builds on (most cited first, 19 of 19)

- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [Neural Probabilistic Logic Programming in DeepProbLog](https://mlsanity.com/paper/1907.08194/) (2019)
- [Logic Tensor Networks](https://mlsanity.com/paper/2012.13635/) (2020)
- [DISC-LawLLM: Fine-tuning Large Language Models for Intelligent Legal Services](https://mlsanity.com/paper/2309.11325/) (2023)
- [Large Language Models as Tax Attorneys: A Case Study in Legal Capabilities Emergence](https://mlsanity.com/paper/2306.07075/) (2023)
- [LeCaRDv2: A Large-Scale Chinese Legal Case Retrieval Dataset](https://mlsanity.com/paper/2310.17609/) (2023)
- [LawLLM: Law Large Language Model for the US Legal System](https://mlsanity.com/paper/2407.21065/) (2024)
- [AgentCourt: Simulating Court with Adversarial Evolvable Lawyer Agents](https://mlsanity.com/paper/2408.08089/) (2024)
- [Unlocking Practical Applications in Legal Domain: Evaluation of GPT for Zero-Shot Semantic Annotation of Legal Texts](https://mlsanity.com/paper/2305.04417/) (2023)
- [Towards Robust Legal Reasoning: Harnessing Logical LLMs in Law](https://mlsanity.com/paper/2502.17638/) (2025)
- [On Verifiable Legal Reasoning: A Multi-Agent Framework with Formalized Knowledge Representations](https://mlsanity.com/paper/2509.00710/) (2025)
- [Explainable Rule Application via Structured Prompting: A Neural-Symbolic Approach](https://mlsanity.com/paper/2506.16335/) (2025)
- [Inherent and emergent liability issues in LLM-based agentic systems: a principal-agent perspective](https://mlsanity.com/paper/2504.03255/) (2025)
- [Neuro-symbolic Learning Yielding Logical Constraints](https://mlsanity.com/paper/2410.20957/) (2024)
- [Enhancing Legal Case Retrieval via Scaling High-quality Synthetic Query-Candidate Pairs](https://mlsanity.com/paper/2410.06581/) (2024)
- [Agents on the Bench: Large Language Model Based Multi Agent Framework for Trustworthy Digital Justice](https://mlsanity.com/paper/2412.18697/) (2024)
- [Incorporating Legal Structure in Retrieval-Augmented Generation: A Case Study on Copyright Fair Use](https://mlsanity.com/paper/2505.02164/) (2025)
- [Logic Rules as Explanations for Legal Case Retrieval](https://mlsanity.com/paper/2403.01457/) (2024)
- [Gaps or Hallucinations? Gazing into Machine-Generated Legal Analysis for Fine-grained Text Evaluations](https://mlsanity.com/paper/2409.09947/) (2024)

## Built on by (most cited first, 5 of 5)

- [LLM Agents in Law: Taxonomy, Applications, and Challenges](https://mlsanity.com/paper/2601.06216/) (2026)
- [Verification Required: The Impact of Information Credibility on AI Persuasion](https://mlsanity.com/paper/2602.00970/) (2026)
- [Formally Verified Patent Analysis via Dependent Type Theory: Machine-Checkable Certificates from a Hybrid AI + Lean 4 Pipeline](https://mlsanity.com/paper/2604.18882/) (2026)
- [REGREACT: Self-Correcting Multi-Agent Pipelines for Structured Regulatory Information Extraction](https://mlsanity.com/paper/2604.12054/) (2026)
- [Probing the Structure and Dynamics of LLM Value Expression through Value Conflicts](https://mlsanity.com/paper/2609.07296/) (2026)

## Abstract (arXiv)

Legal decisions should be logical and based on statutory laws. While large language models(LLMs) are good at understanding legal text, they cannot provide verifiable justifications. We present L4L, a solver-centric framework that enforces formal alignment between LLM-based legal reasoning and statutory laws. The framework integrates role-differentiated LLM agents with SMT-backed verification, combining the flexibility of natural language with the rigor of symbolic reasoning. Our approach operates in four stages: (1) Statute Knowledge Building, where LLMs autoformalize legal provisions into logical constraints and validate them through case-level testing; (2) Dual Fact-and-Statute Extraction, in which the prosecutor-and defense-aligned agents independently map case narratives to argument tuples; (3) Solver-Centric Adjudication, where SMT solvers check the legal admissibility and consistency of the arguments against the formalized statute knowledge; (4) Judicial Rendering, in which a judge agent integrates solver-validated reasoning with statutory interpretation and similar precedents to produce a legally grounded verdict. Experiments on public legal benchmarks show that L4L consistently outperforms baselines, while providing auditable justifications that enable trustworthy legal AI.