# JusticeAxis: Benchmarking Legal Judgment between Rigid Rule Application and Ungrounded Discretion

arXiv: https://arxiv.org/abs/2610.00353 · HTML: https://mlsanity.com/paper/2610.00353/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 24 of its 33 references that are in the map.
- First posted on arXiv 30 Sep 2026.

## Builds on (most cited first, 24 of 24)

- [LegalBench: A Collaboratively Built Benchmark for Measuring Legal Reasoning in Large Language Models](https://mlsanity.com/paper/2308.11462/) (2023)
- [CAIL2018: A Large-Scale Legal Dataset for Judgment Prediction](https://mlsanity.com/paper/1807.02478/) (2018)
- [LawBench: Benchmarking Legal Knowledge of Large Language Models](https://mlsanity.com/paper/2309.16289/) (2023)
- [LegalAgentBench: Evaluating LLM Agents in Legal Domain](https://mlsanity.com/paper/2412.17259/) (2024)
- [LEXam: Benchmarking Legal Reasoning on 340 Law Exams](https://mlsanity.com/paper/2505.12864/) (2025)
- [AgentsCourt: Building Judicial Decision-Making Agents with Court Debate Simulation and Legal Knowledge Augmentation](https://mlsanity.com/paper/2403.02959/) (2024)
- [Legal Syllogism Prompting: Teaching Large Language Models for Legal Judgment Prediction](https://mlsanity.com/paper/2307.08321/) (2023)
- [AgentCourt: Simulating Court with Adversarial Evolvable Lawyer Agents](https://mlsanity.com/paper/2408.08089/) (2024)
- [Can Large Language Models Grasp Legal Theories? Enhance Legal Reasoning with Insights from Multi-Agent Collaboration](https://mlsanity.com/paper/2410.02507/) (2024)
- [Multimodal Multi-Agent Empowered Legal Judgment Prediction](https://mlsanity.com/paper/2601.12815/) (2026)
- [Overview of CAIL2018: Legal Judgment Prediction Competition](https://mlsanity.com/paper/1810.05851/) (2018)
- [From Question Answering to Task Completion: A Survey on Agent System and Harness Design](https://mlsanity.com/paper/2606.20683/) (2026)
- [Large Language Models Meet Legal Artificial Intelligence: A Survey](https://mlsanity.com/paper/2509.09969/) (2025)
- [Chinese Court Simulation with LLM-Based Agent System](https://mlsanity.com/paper/2508.17322/) (2025)
- [A Law Reasoning Benchmark for LLM with Tree-Organized Structures including Factum Probandum, Evidence and Experiences](https://mlsanity.com/paper/2503.00841/) (2025)
- [Challenges for Generative AI in Legal Reasoning](https://mlsanity.com/paper/2508.18880/) (2025)
- [GLARE: Agentic Reasoning for Legal Judgment Prediction](https://mlsanity.com/paper/2508.16383/) (2025)
- [Better Call CLAUSE: A Discrepancy Benchmark for Auditing LLMs Legal Reasoning Capabilities](https://mlsanity.com/paper/2511.00340/) (2025)
- [LLM Agents in Law: Taxonomy, Applications, and Challenges](https://mlsanity.com/paper/2601.06216/) (2026)
- [VERDICT: Verifiable Evolving Reasoning with Directive-Informed Collegial Teams for Legal Judgment Prediction](https://mlsanity.com/paper/2603.19306/) (2026)
- [Magis-Bench: Evaluating LLMs on Magistrate-Level Legal Tasks](https://mlsanity.com/paper/2605.08437/) (2026)
- [EgoPolice: A Benchmark for Egocentric Video Understanding in High-Stakes Police Body-Worn Camera Footage](https://mlsanity.com/paper/2607.06468/) (2026)
- [CrossLex: A Source-Grounded Benchmark for Cross-Jurisdictional Legal Reasoning in Large Language Models](https://mlsanity.com/paper/2608.01292/) (2026)
- [Parthenon Law: A Self-Evolving Legal-Agent Framework](https://mlsanity.com/paper/2606.04602/) (2026)

## Abstract (arXiv)

A sound judgment applies the law to established facts and weighs the circumstances in which they arose. However, existing methods swing between rigid statute matching and ungrounded discretion, benchmarks score a label or a rubric, and the experience that would supply the balance stays unverified. We formalize legal judgment as a reference-anchored task, whose object is a single decision that stays tied to the statute and to the circumstances at once. We introduce JusticeAxis, 256 real-world criminal cases from 18 countries with audio, image, and text evidence, and three lawyer-written judgments for every case: the recorded one and one for each failure. We further propose JusticeAgent, a harness whose element agents establish the facts and whose judge agent applies the law under skills carrying experience of the circumstances. Skills are distilled from execution trajectories and admitted only under Bayesian credible bounds. Experiments show that failure turns direction with scale: open-weight backbones drift to unsupported grounds, frontier models to the statutory default. We further verify that JusticeAgent, as a simple yet effective plugin, carries a frozen open-weight backbone to commercial level. Project resources are available at https://github.com/beita6969/JusticeAxis.