# When Fairness Isn't Statistical: The Limits of Machine Learning in Evaluating Legal Reasoning

arXiv: https://arxiv.org/abs/2506.03913 · HTML: https://mlsanity.com/paper/2506.03913/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 22 references that are in the map.
- First posted on arXiv 4 Jun 2025.

## Builds on (most cited first, 5 of 5)

- [Equality of Opportunity in Supervised Learning](https://mlsanity.com/paper/1610.02413/) (2016)
- [Problem Formulation and Fairness](https://mlsanity.com/paper/1901.02547/) (2019)
- [Changing the World by Changing the Data](https://mlsanity.com/paper/2105.13947/) (2021)
- [Fairness and Explainability: Bridging the Gap Towards Fair Model Explanations](https://mlsanity.com/paper/2212.03840/) (2022)
- [Equality before the Law: Legal Judgment Consistency Analysis for Fairness](https://mlsanity.com/paper/2103.13868/) (2021)

## Built on by (most cited first, 2 of 2)

- [Evaluation Cards: An Interpretive Layer for AI Evaluation Reporting](https://mlsanity.com/paper/2606.09809/) (2026)
- [The Judge Variable: Challenging Judge-Agnostic Legal Judgment Prediction](https://mlsanity.com/paper/2507.13732/) (2025)

## Abstract (arXiv)

Legal decisions are increasingly evaluated for fairness, consistency, and bias using machine learning (ML) techniques. In high-stakes domains like refugee adjudication, such methods are often applied to detect disparities in outcomes. Yet it remains unclear whether statistical methods can meaningfully assess fairness in legal contexts shaped by discretion, normative complexity, and limited ground truth. In this paper, we empirically evaluate three common ML approaches (feature-based analysis, semantic clustering, and predictive modeling) on a large, real-world dataset of 59,000+ Canadian refugee decisions (AsyLex). Our experiments show that these methods produce divergent and sometimes contradictory signals, that predictive modeling often depends on contextual and procedural features rather than legal features, and that semantic clustering fails to capture substantive legal reasoning. We show limitations of statistical fairness evaluation, challenge the assumption that statistical regularity equates to fairness, and argue that current computational approaches fall short of evaluating fairness in legally discretionary domains. We argue that evaluating fairness in law requires methods grounded not only in data, but in legal reasoning and institutional context.