# Agents on the Bench: Large Language Model Based Multi Agent Framework for Trustworthy Digital Justice

arXiv: https://arxiv.org/abs/2412.18697 · HTML: https://mlsanity.com/paper/2412.18697/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Placed in Social simulation and multi-agent debate, one of 16 directions in LLM agents and tool use.
- Builds on 16 of its 28 references that are in the map.
- First posted on arXiv 24 Dec 2024.

## Placed in

- LLM agents and tool use / [Social simulation and multi-agent debate](https://mlsanity.com/agents/social-simulation-and-multi-agent-debate/)

## Builds on (most cited first, 16 of 16)

- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [Generative Agents: Interactive Simulacra of Human Behavior](https://mlsanity.com/paper/2304.03442/) (2023)
- [MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework](https://mlsanity.com/paper/2308.00352/) (2023)
- [ChatDev: Communicative Agents for Software Development](https://mlsanity.com/paper/2307.07924/) (2023)
- [Agent Hospital: A Simulacrum of Hospital with Evolvable Medical Agents](https://mlsanity.com/paper/2405.02957/) (2024)
- [Neural Legal Judgment Prediction in English](https://mlsanity.com/paper/1906.02059/) (2019)
- [CAIL2018: A Large-Scale Legal Dataset for Judgment Prediction](https://mlsanity.com/paper/1807.02478/) (2018)
- [Can GPT-3 Perform Statutory Reasoning?](https://mlsanity.com/paper/2302.06100/) (2023)
- [Learning to Predict Charges for Criminal Cases with Legal Basis](https://mlsanity.com/paper/1707.09168/) (2017)
- [A Survey on Legal Judgment Prediction: Datasets, Metrics, Models and Challenges](https://mlsanity.com/paper/2204.04859/) (2022)
- [Blind Judgement: Agent-Based Supreme Court Modelling With GPT](https://mlsanity.com/paper/2301.05327/) (2023)
- [Legal Syllogism Prompting: Teaching Large Language Models for Legal Judgment Prediction](https://mlsanity.com/paper/2307.08321/) (2023)
- [Leveraging Large Language Models for Learning Complex Legal Concepts through Storytelling](https://mlsanity.com/paper/2402.17019/) (2024)
- [Applicability of Large Language Models and Generative Models for Legal Case Judgement Summarization](https://mlsanity.com/paper/2407.12848/) (2024)
- [Through the Lens of Split Vote: Exploring Disagreement, Difficulty and Calibration in Legal Case Outcome Classification](https://mlsanity.com/paper/2402.07214/) (2024)
- [Do Charge Prediction Models Learn Legal Theory?](https://mlsanity.com/paper/2210.17108/) (2022)

## Built on by (most cited first, 10 of 10)

- [LLM Agents for Education: Advances and Applications](https://mlsanity.com/paper/2503.11733/) (2025)
- [Towards Trustworthy Legal AI through LLM Agents and Formal Reasoning](https://mlsanity.com/paper/2511.21033/) (2025)
- [SAMVAD: A Multi-Agent System for Simulating Judicial Deliberation Dynamics in India](https://mlsanity.com/paper/2509.03793/) (2025)
- [Law in Silico: Simulating Legal Society with LLM-Based Agents](https://mlsanity.com/paper/2510.24442/) (2025)
- [Mina: A Multilingual LLM-Powered Legal Assistant Agent for Bangladesh for Empowering Access to Justice](https://mlsanity.com/paper/2511.08605/) (2025)
- [AEGIS: From Clues to Verdicts -- Graph-Guided Deep Vulnerability Reasoning via Dialectics and Meta-Auditing](https://mlsanity.com/paper/2603.20637/) (2026)
- [Legal Document Summarization: Enhancing Judicial Efficiency through Automation Detection](https://mlsanity.com/paper/2507.18952/) (2025)
- [Which Changes Matter? Towards Trustworthy Legal AI via Relevance-Sensitive Evaluation and Solver-Grounded Reasoning](https://mlsanity.com/paper/2605.26530/) (2026)
- [Investigating Multi-Agent Deliberation in Law](https://mlsanity.com/paper/2606.30906/) (2026)
- [From Process Loss to Assembly Bonus: Human-Grounded Diagnosis of Multi-Agent LLM Collaboration](https://mlsanity.com/paper/2609.13261/) (2026)

## Abstract (arXiv)

The justice system has increasingly employed AI techniques to enhance efficiency, yet limitations remain in improving the quality of decision-making, particularly regarding transparency and explainability needed to uphold public trust in legal AI. To address these challenges, we propose a large language model based multi-agent framework named AgentsBench, which aims to simultaneously improve both efficiency and quality in judicial decision-making. Our approach leverages multiple LLM-driven agents that simulate the collaborative deliberation and decision making process of a judicial bench. We conducted experiments on legal judgment prediction task, and the results show that our framework outperforms existing LLM based methods in terms of performance and decision quality. By incorporating these elements, our framework reflects real-world judicial processes more closely, enhancing accuracy, fairness, and society consideration. AgentsBench provides a more nuanced and realistic methods of trustworthy AI decision-making, with strong potential for application across various case types and legal scenarios.