mlsanityData to 7 Oct 2026

Does the Competitive Component of Adversarial Self-Play Improve Legal Reasoning? A Controlled Negative Result

arXiv 2608.01559 ↗First posted 3 Aug 2026cs.AI cs.CL cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Adversarial self-play is an appealing recipe for legal reasoning: have a student model draft an argument, have an adversary attack it, and reward the student when its argument survives the attack. We designed exactly such a training signal -- a verifiable "survival" reward in which both the student's cited authorities and the adversary's counter-authorities are checked by a citation verifier, so that survival is decided on verified grounds rather than rhetoric, and fabricated citations are automatically neutralized. We then asked a narrow but important question: does the competitive component itself -- the adversary and the survival reward -- add anything on top of an otherwise identical non-competitive training run? Across four independent tests -- a bootstrap comparison, a two-seed replication, a paired per-case adversarial-robustness comparison, and a blinded head-to-head judgment of generated arguments, plus a follow-up pilot with a deliberately strengthened self-play adversary -- the competitive component produced no reliable benefit. The blinded judgment gave a 49% win rate (binomial p approx. 1.000); the strengthened-adversary pilot gave a 50% win rate (32:32, p approx. 1.000). An early apparent +29% advantage reversed and proved to be a small-sample artifact. We report this as an honest negative result. The value of the paper is reproducibility and the sharing of concrete pitfalls: an initially promising metric that inverted on more data, and an adversarial-robustness metric that silently collapsed to plain recall once the adversary stopped citing the same authorities as the gold answer. This null is consistent with, and reconfirms in the legal domain, the conclusion of the companion coding-domain study (Kim, 2026, arXiv:2607.08255) that the value of multi-teacher curricula arises from constructing a verifiable environment rather than from competition itself.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 9 of its 11 references that are in the map. First posted on arXiv 3 Aug 2026.

Builds on

9 of 11 references.

References · 9

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  2. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  3. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  4. 2019Adversarial NLI: A New Benchmark for Natural Language Understandingcited by 798
  5. 2018AI safety via debatecited by 384
  6. 2024AgentCourt: Simulating Court with Adversarial Evolvable Lawyer Agentscited by 36
  7. 2026Courtroom-Style Multi-Agent Debate with Progressive RAG and Role-Switching for Controversial Claim Verificationcited by 4
  8. 2025ASP2LJ : An Adversarial Self-Play Laywer Augmented Legal Judgment Frameworkcited by 3
  9. 2026Compete Then Collaborate: Frontier AI Teachers Build a Verifiable Curriculum to Improve a Coding Student Beyond Imitationcited by 2

Built on by

Not cited yet.