# AI-Assisted Moot Courts: Simulating Justice-Specific Questioning in Oral Arguments

arXiv: https://arxiv.org/abs/2603.04718 · HTML: https://mlsanity.com/paper/2603.04718/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 78 references that are in the map.
- First posted on arXiv 5 Mar 2026.

## Builds on (most cited first, 23 of 23)

- [Towards Understanding Sycophancy in Language Models](https://mlsanity.com/paper/2310.13548/) (2023)
- [CUAD: An Expert-Annotated NLP Dataset for Legal Contract Review](https://mlsanity.com/paper/2103.06268/) (2021)
- [Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools](https://mlsanity.com/paper/2405.20362/) (2024)
- [LegalBench-RAG: A Benchmark for Retrieval-Augmented Generation in the Legal Domain](https://mlsanity.com/paper/2408.10343/) (2024)
- [Question Answering for Privacy Policies: Combining Computational and Legal Perspectives](https://mlsanity.com/paper/1911.00841/) (2019)
- [A Dataset for Statutory Reasoning in Tax Law Entailment and Question Answering](https://mlsanity.com/paper/2005.05257/) (2020)
- [Towards Responsible Development of Generative AI for Education: An Evaluation-Driven Approach](https://mlsanity.com/paper/2407.12687/) (2024)
- [A Survey on Legal Judgment Prediction: Datasets, Metrics, Models and Challenges](https://mlsanity.com/paper/2204.04859/) (2022)
- [Tutor CoPilot: A Human-AI Approach for Scaling Real-Time Expertise](https://mlsanity.com/paper/2410.03017/) (2024)
- [AgentCourt: Simulating Court with Adversarial Evolvable Lawyer Agents](https://mlsanity.com/paper/2408.08089/) (2024)
- [A Reasoning-Focused Legal Retrieval Benchmark](https://mlsanity.com/paper/2505.03970/) (2025)
- [Modeling Legal Reasoning: LM Annotation at the Edge of Human Agreement](https://mlsanity.com/paper/2310.18440/) (2023)
- [Can LLM-Simulated Practice and Feedback Upskill Human Counselors? A Randomized Study with 90+ Novice Counselors](https://mlsanity.com/paper/2505.02428/) (2025)
- [Evaluating AI for Law: Bridging the Gap with Open-Source Solutions](https://mlsanity.com/paper/2404.12349/) (2024)
- [A Survey on Legal Question Answering Systems](https://mlsanity.com/paper/2110.07333/) (2021)
- [ContractEval: Benchmarking LLMs for Clause-Level Legal Risk Identification in Commercial Contracts](https://mlsanity.com/paper/2508.03080/) (2025)
- [Chinese Court Simulation with LLM-Based Agent System](https://mlsanity.com/paper/2508.17322/) (2025)
- [Generative AI in the Classroom: Can Students Remain Active Learners?](https://mlsanity.com/paper/2310.03192/) (2023)
- [LAR-ECHR: A New Legal Argument Reasoning Task and Dataset for Cases of the European Court of Human Rights](https://mlsanity.com/paper/2410.13352/) (2024)
- [FALQU: Finding Answers to Legal Questions](https://mlsanity.com/paper/2304.05611/) (2023)
- [A Computational Analysis of Oral Argument in the Supreme Court](https://mlsanity.com/paper/2306.05373/) (2023)
- [How Can AI Augment Access to Justice? Public Defenders' Perspectives on AI Adoption](https://mlsanity.com/paper/2510.22933/) (2025)
- [Dewey Long Context Embedding Model: A Technical Report](https://mlsanity.com/paper/2503.20376/) (2025)

## Built on by (most cited first, 1 of 1)

- [Reading Between The Lines: Modeling and Evaluating Behavioral Realism in Legal Simulation](https://mlsanity.com/paper/2608.13712/) (2026)

## Abstract (arXiv)

In oral arguments, judges probe attorneys with questions about the factual record, legal claims, and the strength of their arguments. To prepare for this questioning, both law schools and practicing attorneys rely on moot courts: practice simulations of appellate hearings. Leveraging a dataset of U.S. Supreme Court oral argument transcripts, we examine whether AI models can effectively simulate justice-specific questioning for moot court-style training. Evaluating oral argument simulation is challenging because there is no single correct question for any given turn. Instead, effective questioning should reflect a combination of desirable qualities, such as anticipating substantive legal issues, detecting logical weaknesses, and maintaining an appropriately adversarial tone. We introduce a two-layer evaluation framework that assesses both the realism and pedagogical usefulness of simulated questions using complementary proxy metrics. We construct and evaluate both prompt-based and agentic oral argument simulators. We find that simulated questions are often perceived as realistic by human annotators and achieve high recall of ground truth substantive legal issues. However, models still face substantial shortcomings, including low diversity in question types and sycophancy. Importantly, these shortcomings would remain undetected under naive evaluation approaches.