mlsanityData to 7 Oct 2026

GRACE: Grounded Adversarial Reasoning over Canadian Law

arXiv 2609.23726 ↗First posted 20 Sep 2026cs.CL cs.AIcited by 0

On the map: LLM reasoning → Reasoning benchmarks and evaluation also fits: Reasoning for agents, planning and code

Abstract (arXiv)

Large language models have shown strong performance across a range of legal tasks, but existing benchmarks rarely evaluate the ability to take and defend a legal position, reason under incomplete information, or synthesize multiple statutory provisions. This gap is particularly pronounced for Canadian law, which remains underrepresented in legal NLP. We introduce GRACE (Grounded Reasoning Adversarial Canadian LEgal examples), a dataset of 1,915 question-reasoning-answer instances grounded in Canadian federal legislation. GRACE covers three reasoning modes: adversarial advocacy, uncertainty, and applied reasoning. We develop a pipeline that partitions raw statutory text, generates scenario-based questions and reasoning, and filters examples through model-free citation verification and LLM-based quality auditing. As a proof of concept, we fine-tune CLeAR-4B (Canadian Legal Adversarial Reasoning), a lightweight model for grounded legal reasoning, and evaluate it against the unmodified Qwen3-4B base model in open- and closed-book settings. CLeAR-4B substantially improves agreement with teacher outputs and statutory citation behavior when the relevant act text is provided, while its grounding degrades sharply when the statute is withheld. These results suggest that GRACE can support the development of lightweight legal models that reason more effectively from supplied statutory text.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 7 of its 16 references that are in the map. First posted on arXiv 20 Sep 2026.

Builds on

7 of 16 references.

References · 7

  1. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  2. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  3. 2024SaulLM-7B: A pioneering Large Language Model for Lawcited by 136
  4. 2023Chatlaw: A Multi-Agent Legal Assistant based on a Role-Aligned Mixture-of-Experts Architecturecited by 119
  5. 2023DISC-LawLLM: Fine-tuning Large Language Models for Intelligent Legal Servicescited by 117
  6. 2025Introducing the A2AJ's Canadian Legal Data: An open-source alternative to CanLII for the era of computational lawcited by 2
  7. 2026CanLegalRAGBench: Evaluating Retrieval-Augmented Generation on Canadian Case Lawcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Evaluating the Limits of Large Language Models in Multilingual Legal Reasoning
  2. 2022The Legal Argument Reasoning Task in Civil Procedure
  3. 2025Evaluating Test-Time Scaling LLMs for Legal Reasoning: OpenAI o1, DeepSeek-R1, and Beyond
  4. 2026Can Small Models Reason About Legal Documents? A Comparative Study
  5. 2026LePREC: Reasoning as Classification over Structured Factors for Assessing Relevance of Legal Issues
  6. 2026Multi-Legal-Bench: Evaluating LLMs on Legal Reasoning Across Jurisdictions, Languages, and Legal Traditions
  7. 2026Building Legal Reward Models for Grounding and Abstention
  8. 2026LabourCrew: A Multi-Agent RAG Framework for Trustworthy Adversarial Deliberation and Statutory Reasoning over Labour Law