mlsanityData to 14 Oct 2026

ImmigrationReason: A Structured Dataset of U.S. Immigration Appeals for Legal Reasoning Research

arXiv 2608.20391 ↗First posted 30 Jun 2026cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Most legal NLP resources draw from federal case law and focus on coarse classification, leaving administrative adjudication, where the vast majority of government decisions occur, essentially unaddressed. We introduce ImmigrationReason, a large-scale structured dataset derived from 12,375 non-precedent decisions of the U.S. Citizenship and Immigration Services (USCIS) Administrative Appeals Office (AAO) spanning 2005 to 2026. Each record captures the applicable legal framework, per-criterion evidence-sufficiency findings under a five-category label, verbatim adjudicator-criticism quotes, all citations, and final dispositions, alongside high-quality Claude-transcribed source text. Extraction quality is validated through a three-pass pipeline combining two independent modalities with comparison-prompt adjudication by Opus 4.7, and verified by domain experts on a 500-record sample. The dataset documents nearly 9,000 verbatim instances of AAO-identified legal errors, spans a natural legal-regime transition (the 2016 Dhanasar rule change), and covers 21 years of adjudication. We analyze the dataset in detail and outline research directions it enables, from outcome prediction and adjudicator-error analysis to agent design for high-stakes regulatory domains.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 8 of its 22 references that are in the map. First posted on arXiv 30 Jun 2026.

Builds on

8 of 22 references.

References · 8

  1. 2023LegalBench: A Collaboratively Built Benchmark for Measuring Legal Reasoning in Large Language Modelscited by 336
  2. 2021LexGLUE: A Benchmark Dataset for Legal Language Understanding in Englishcited by 310
  3. 2024Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Toolscited by 176
  4. 2024Large Legal Fictions: Profiling Legal Hallucinations in Large Language Modelscited by 150
  5. 2023LawBench: Benchmarking Legal Knowledge of Large Language Modelscited by 117
  6. 2022Pile of Law: Learning Responsible Data Filtering from the Law and a 256GB Open-Source Legal Datasetcited by 113
  7. 2024Croissant: A Metadata Format for ML-Ready Datasetscited by 81
  8. 2023Empowering Refugee Claimants and their Lawyers: Using Machine Learning to Examine Decision-Making in Refugee Lawcited by 1

Built on by

Not cited yet.