mlsanityData to 7 Oct 2026

Blind Judgement: Agent-Based Supreme Court Modelling With GPT

arXiv 2301.05327 ↗First posted 12 Jan 2023cs.CLcited by 37

On the map: LLM agents and tool use → Social simulation and multi-agent debate

Abstract (arXiv)

We present a novel Transformer-based multi-agent system for simulating the judicial rulings of the 2010-2016 Supreme Court of the United States. We train nine separate models with the respective authored opinions of each supreme justice active ca. 2015 and test the resulting system on 96 real-world cases. We find our system predicts the decisions of the real-world Supreme Court with better-than-random accuracy. We further find a correlation between model accuracy with respect to individual justices and their alignment between legal conservatism & liberalism. Our methods and results hold significance for researchers interested in using language models to simulate politically-charged discourse between multiple agents.

Key facts. Cited by 37 papers in the map (data through 7 Oct 2026). Placed in Social simulation and multi-agent debate, one of 16 directions in LLM agents and tool use. Builds on 15 of its 28 references that are in the map. First posted on arXiv 12 Jan 2023.

Builds on

15 of 28 references.

References · 15

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  5. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  6. 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 881
  7. 2022Out of One, Many: Using Language Models to Simulate Human Samplescited by 737
  8. 2022GPT-NeoX-20B: An Open-Source Autoregressive Language Modelcited by 711
  9. 2018Object Hallucination in Image Captioningcited by 661
  10. 2022Social Simulacra: Creating Populated Prototypes for Social Computing Systemscited by 300
  11. 2022Scaling Up Models and Data with textttt5x and textttseqiocited by 187
  12. 2021A Neural Network Solves, Explains, and Generates University Math Problems by Program Synthesis and Few-Shot Learning at Human Levelcited by 96
3 more
  1. 2022Generalizing over Long Tail Concepts for Medical Term Normalizationcited by 8
  2. 2021Natural-Language Multi-Agent Simulations of Argumentative Opinion Dynamicscited by 5
  3. 2022The COVID That Wasn't: Counterfactual Journalism Using GPTcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 14; 2024: 14; 2025: 5; 2026: 4.

37 papers.

Sort:

By area:

  1. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,203
  2. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,422
  3. 2023Multi-Agent Collaboration: Harnessing the Power of Intelligent LLM Agentscited by 329
  4. 2023Challenges and Applications of Large Language Modelscited by 251
  5. 2023CoMPosT: Characterizing and Evaluating Caricature in LLM Simulationscited by 94
  6. 2025Lifelong Learning of Large Language Model based Agents: A Roadmapcited by 78
  7. 2024From Individual to Society: A Survey on Social Simulation Driven by Large Language Model-based Agentscited by 76
  8. 2024Fairness in Large Language Models: A Taxonomic Surveycited by 57
  9. 2024Operationalizing a Threat Model for Red-Teaming Large Language Models (LLMs)cited by 44
  10. 2023Explaining Legal Concepts with Augmented Large Language Models (GPT-4)cited by 39
  11. 2024AgentCourt: Simulating Court with Adversarial Evolvable Lawyer Agentscited by 36
  12. 2024Agent Design Pattern Catalogue: A Collection of Architectural Patterns for Foundation Model based Agentscited by 35
25 more
  1. 2023Unlocking Practical Applications in Legal Domain: Evaluation of GPT for Zero-Shot Semantic Annotation of Legal Textscited by 32
  2. 2023Can GPT-4 Support Analysis of Textual Data in Tasks Requiring Highly Specialized Domain Expertise?cited by 30
  3. 2024Simulating Human Strategic Behavior: Comparing Single and Multi-agent LLMscited by 20
  4. 2023Generative Agent-Based Modeling: Unveiling Social System Dynamics through Coupling Mechanistic Models with Generative Artificial Intelligencecited by 19
  5. 2024LLM-based multi-agent poetry generation in non-cooperative environmentscited by 15
  6. 2024An Electoral Approach to Diversify LLM-based Multi-Agent Collective Decision-Makingcited by 14
  7. 2023Towards Responsible Generative AI: A Reference Architecture for Designing Foundation Model based Agentscited by 13
  8. 2024DEEM: Dynamic Experienced Expert Modeling for Stance Detectioncited by 12
  9. 2023Smart Agent-Based Modeling: On the Use of Large Language Models in Computer Simulationscited by 11
  10. 2024Agents on the Bench: Large Language Model Based Multi Agent Framework for Trustworthy Digital Justicecited by 10
  11. 2024BenchmarkCards: Standardized Documentation for Large Language Model Benchmarkscited by 8
  12. 2023Algorithmic Ghost in the Research Shell: Large Language Models and Academic Knowledge Creation in Management Researchcited by 6
  13. 2023LLMediator: GPT-4 Assisted Online Dispute Resolutioncited by 6
  14. 2024It Cannot Be Right If It Was Written by AI: On Lawyers' Preferences of Documents Perceived as Authored by an LLM vs a Humancited by 5
  15. 2025SAMVAD: A Multi-Agent System for Simulating Judicial Deliberation Dynamics in Indiacited by 4
  16. 2024Analyzing Images of Legal Documents: Toward Multi-Modal LLMs for Access to Justicecited by 3
  17. 2024Synthetic Social Media Influence Experimentation via an Agentic Reinforcement Learning Large Language Model Botcited by 2
  18. 2025DoubleAgents: Human-Agent Alignment in a Socially Embedded Workflowcited by 1
  19. 2026Gender Bias in LLMs: Preliminary Evidence from Shared Parenting Scenario in Czech Family Lawcited by 1
  20. 2026Interpretable Traffic Responsibility from Dashcam Video via Legal Multi Agent Reasoningcited by 1
  21. 2023Can Generative Pre-trained Transformers (GPT) Pass Assessments in Higher Education Programming Courses?cited by 0
  22. 2025Large Language Model-Based Agents for Automated Research Reproducibility: An Exploratory Study in Alzheimer's Diseasecited by 0
  23. 2025Skim-Aware Contrastive Learning for Efficient Document Representationcited by 0
  24. 2026Strategic Persuasion with Trait-Conditioned Multi-Agent Systems for Iterative Legal Argumentationcited by 0
  25. 2026Prompt Sensitivity of Generative Agents: Evidence from an Epidemic Modelcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024AgentsCourt: Building Judicial Decision-Making Agents with Court Debate Simulation and Legal Knowledge Augmentation
  2. 2026Investigating Multi-Agent Deliberation in Law
  3. 2025Debate-Feedback: A Multi-Agent Framework for Efficient Legal Judgment Prediction
  4. 2024Rethinking Legal Judgement Prediction in a Realistic Scenario in the Era of Large Language Models
  5. 2025When AIs Judge AIs: The Rise of Agent-as-a-Judge Evaluation for LLMs
  6. 2026Courtroom-Style Multi-Agent Debate with Progressive RAG and Role-Switching for Controversial Claim Verification
  7. 2026L-MAD: A Systematic Evaluation of Multi-Agent Debate Structures in Legal Reasoning
  8. 2024Can Large Language Models Grasp Legal Theories? Enhance Legal Reasoning with Insights from Multi-Agent Collaboration