mlsanityData to 7 Oct 2026

AI Achieves a Perfect LSAT Score

arXiv 2604.10034 ↗First posted 11 Apr 2026cs.AIcited by 0

On the map: LLM reasoning → Mathematical reasoning

Abstract (arXiv)

This paper reports the first documented instance of a language model achieving a perfect score on an officially disclosed Law School Admission Test (LSAT). Controlled experiments on eight reasoning models show that varying the prompt, shuffling answer choices, and sampling multiple responses have no meaningful effect as drivers of performance. Ablating the thinking phase that models generate before answering, however, lowers frontier accuracy by up to 8 percentage points, predominantly in logical reasoning. Distilled models produce full thinking traces in the same format yet plateau far below frontier performance. A pilot process reward model fine-tuned via QLoRA on official LSAT explanations narrows this gap through Best-of-5 selection, with gains again predominantly in logical reasoning. The gatekeeper of elite legal education since 1948, the LSAT has not merely been passed but answered without a single error by models that reason. The upper bound of the cognitive capacities it has tested is no longer exclusive to human cognition.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Mathematical reasoning, one of 17 directions in LLM reasoning. Builds on 25 of its 61 references that are in the map. First posted on arXiv 11 Apr 2026.

Builds on

25 of 61 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models5 Feb

Other references · 24

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  3. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  4. 2020Scaling Laws for Neural Language Modelscited by 6,411
  5. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  6. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  7. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
  8. 2023Let's Verify Step by Stepcited by 3,598
  9. 2022Training Compute-Optimal Large Language Modelscited by 2,927
  10. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,928
  11. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,371
  12. 2025s1: Simple test-time scalingcited by 1,311
12 more
  1. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,193
  2. 2021Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivitycited by 1,122
  3. 2022STaR: Bootstrapping Reasoning With Reasoningcited by 895
  4. 2022Solving math word problems with process- and outcome-based feedbackcited by 871
  5. 2023Large Language Models Are Not Robust Multiple Choice Selectorscited by 454
  6. 2025L1: Controlling How Long A Reasoning Model Thinks With Reinforcement Learningcited by 381
  7. 2023Towards Revealing the Mystery behind Chain of Thought: A Theoretical Perspectivecited by 293
  8. 2024Quiet-STaR: Language Models Can Teach Themselves to Think Before Speakingcited by 292
  9. 2023Think before you speak: Training Language Models With Pause Tokenscited by 254
  10. 2023Large Language Models Sensitivity to The Order of Options in Multiple-Choice Questionscited by 249
  11. 2025Know What You Don't Know: Uncertainty Calibration of Process Reward Modelscited by 19
  12. 2016On the Diagnostic of Road Pathway Visibilitycited by 10

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Lost in the Logic: An Evaluation of Large Language Models' Reasoning Capabilities on LSAT Logic Games
  2. 2026Better Call Reward: Reward Hacking as Strategic Abstention in Legal Reasoning Models
  3. 2026The IOL-AI Challenge: An Open Challenge towards Advancing Linguistic Reasoning
  4. 2025From Test-taking to Cognitive Scaffolding: A Pedagogical Diagnostic Benchmark for LLMs on English Standardized Tests
  5. 2026Catching The Correct Answer Trap: Characterising AI Tutor Blind Spots When Analysing Student Reasoning
  6. 2026AI-Assisted Moot Courts: Simulating Justice-Specific Questioning in Oral Arguments
  7. 2025Explicit Reasoning Makes Better Judges: A Systematic Study on Accuracy, Efficiency, and Robustness
  8. 2026RealMath-Eval: Why SOTA Judges Struggle with Real Human Reasoning