mlsanityData to 7 Oct 2026

JusticeAxis: Benchmarking Legal Judgment between Rigid Rule Application and Ungrounded Discretion

arXiv 2610.00353 ↗First posted 30 Sep 2026cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

A sound judgment applies the law to established facts and weighs the circumstances in which they arose. However, existing methods swing between rigid statute matching and ungrounded discretion, benchmarks score a label or a rubric, and the experience that would supply the balance stays unverified. We formalize legal judgment as a reference-anchored task, whose object is a single decision that stays tied to the statute and to the circumstances at once. We introduce JusticeAxis, 256 real-world criminal cases from 18 countries with audio, image, and text evidence, and three lawyer-written judgments for every case: the recorded one and one for each failure. We further propose JusticeAgent, a harness whose element agents establish the facts and whose judge agent applies the law under skills carrying experience of the circumstances. Skills are distilled from execution trajectories and admitted only under Bayesian credible bounds. Experiments show that failure turns direction with scale: open-weight backbones drift to unsupported grounds, frontier models to the statutory default. We further verify that JusticeAgent, as a simple yet effective plugin, carries a frozen open-weight backbone to commercial level. Project resources are available at https://github.com/beita6969/JusticeAxis.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 24 of its 33 references that are in the map. First posted on arXiv 30 Sep 2026.

Builds on

24 of 33 references.

References · 24

  1. 2023LegalBench: A Collaboratively Built Benchmark for Measuring Legal Reasoning in Large Language Modelscited by 328
  2. 2018CAIL2018: A Large-Scale Legal Dataset for Judgment Predictioncited by 168
  3. 2023LawBench: Benchmarking Legal Knowledge of Large Language Modelscited by 111
  4. 2024LegalAgentBench: Evaluating LLM Agents in Legal Domaincited by 56
  5. 2025LEXam: Benchmarking Legal Reasoning on 340 Law Examscited by 52
  6. 2024AgentsCourt: Building Judicial Decision-Making Agents with Court Debate Simulation and Legal Knowledge Augmentationcited by 39
  7. 2023Legal Syllogism Prompting: Teaching Large Language Models for Legal Judgment Predictioncited by 37
  8. 2024AgentCourt: Simulating Court with Adversarial Evolvable Lawyer Agentscited by 36
  9. 2024Can Large Language Models Grasp Legal Theories? Enhance Legal Reasoning with Insights from Multi-Agent Collaborationcited by 14
  10. 2026Multimodal Multi-Agent Empowered Legal Judgment Predictioncited by 14
  11. 2018Overview of CAIL2018: Legal Judgment Prediction Competitioncited by 13
  12. 2025Large Language Models Meet Legal Artificial Intelligence: A Surveycited by 9
12 more
  1. 2026From Question Answering to Task Completion: A Survey on Agent System and Harness Designcited by 9
  2. 2025A Law Reasoning Benchmark for LLM with Tree-Organized Structures including Factum Probandum, Evidence and Experiencescited by 7
  3. 2025Chinese Court Simulation with LLM-Based Agent Systemcited by 7
  4. 2025Challenges for Generative AI in Legal Reasoningcited by 7
  5. 2025GLARE: Agentic Reasoning for Legal Judgment Predictioncited by 6
  6. 2025Better Call CLAUSE: A Discrepancy Benchmark for Auditing LLMs Legal Reasoning Capabilitiescited by 6
  7. 2026LLM Agents in Law: Taxonomy, Applications, and Challengescited by 5
  8. 2026VERDICT: Verifiable Evolving Reasoning with Directive-Informed Collegial Teams for Legal Judgment Predictioncited by 2
  9. 2026EgoPolice: A Benchmark for Egocentric Video Understanding in High-Stakes Police Body-Worn Camera Footagecited by 2
  10. 2026Magis-Bench: Evaluating LLMs on Magistrate-Level Legal Taskscited by 1
  11. 2026Parthenon Law: A Self-Evolving Legal-Agent Frameworkcited by 1
  12. 2026CrossLex: A Source-Grounded Benchmark for Cross-Jurisdictional Legal Reasoning in Large Language Modelscited by 1

Built on by

Not cited yet.