# Where Experts Disagree, Models Fail: Detecting Implicit Legal Citations in French Court Decisions

arXiv: https://arxiv.org/abs/2603.22973 · HTML: https://mlsanity.com/paper/2603.22973/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 44 references that are in the map.
- First posted on arXiv 24 Mar 2026.

## Builds on (most cited first, 20 of 20)

- [Billion-scale similarity search with GPUs](https://mlsanity.com/paper/1702.08734/) (2017)
- [Gemma 2: Improving Open Language Models at a Practical Size](https://mlsanity.com/paper/2408.00118/) (2024)
- [Don't Stop Pretraining: Adapt Language Models to Domains and Tasks](https://mlsanity.com/paper/2004.10964/) (2020)
- [MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers](https://mlsanity.com/paper/2002.10957/) (2020)
- [M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillation](https://mlsanity.com/paper/2402.03216/) (2024)
- [MPNet: Masked and Permuted Pre-training for Language Understanding](https://mlsanity.com/paper/2004.09297/) (2020)
- [CamemBERT: a Tasty French Language Model](https://mlsanity.com/paper/1911.03894/) (2019)
- [Efficient Natural Language Response Suggestion for Smart Reply](https://mlsanity.com/paper/1705.00652/) (2017)
- [SaulLM-7B: A pioneering Large Language Model for Law](https://mlsanity.com/paper/2403.03883/) (2024)
- [Toward a Perspectivist Turn in Ground Truthing for Predictive Computing](https://mlsanity.com/paper/2109.04270/) (2021)
- [A Dataset for Statutory Reasoning in Tax Law Entailment and Question Answering](https://mlsanity.com/paper/2005.05257/) (2020)
- [SaulLM-54B & SaulLM-141B: Scaling Up Domain Adaptation for the Legal Domain](https://mlsanity.com/paper/2407.19584/) (2024)
- [Crowdsourcing and Validating Event-focused Emotion Corpora for German and English](https://mlsanity.com/paper/1905.13618/) (2019)
- [LeSICiN: A Heterogeneous Graph-based Approach for Automatic Legal Statute Identification from Indian Legal Documents](https://mlsanity.com/paper/2112.14731/) (2021)
- [U-CREAT: Unsupervised Case Retrieval using Events extrAcTion](https://mlsanity.com/paper/2307.05260/) (2023)
- [JuriBERT: A Masked-Language Model Adaptation for French Legal Text](https://mlsanity.com/paper/2110.01485/) (2021)
- [CamemBERT 2.0: A Smarter French Language Model Aged to Perfection](https://mlsanity.com/paper/2411.08868/) (2024)
- [Entailed Between the Lines: Incorporating Implication into NLI](https://mlsanity.com/paper/2501.07719/) (2025)
- [Lawma: The Power of Specialization for Legal Annotation](https://mlsanity.com/paper/2407.16615/) (2024)
- [IL-PCSR: Legal Corpus for Prior Case and Statute Retrieval](https://mlsanity.com/paper/2511.00268/) (2025)

## Built on by (most cited first, 1 of 1)

- [Classifying Interpretive Canons at the Sentence Level: A Benchmark from the German Federal Constitutional Court](https://mlsanity.com/paper/2609.26945/) (2026)

## Abstract (arXiv)

Applying computational methods to law at scale requires separating genuine legal reasoning from surface similarity. We study this through a concrete task: detecting implicit citations of the French Civil Code, where a court applies a statutory rule without naming it (a post-hoc question about the reasoning a court actually used). We release a benchmark of 1,015 passage-article pairs annotated by three legal experts. Our central finding is that their disagreement is itself informative: the third of cases the experts dispute are where models fail. Our best ensemble reaches an F1 score of 0.70 overall. Yet, two-thirds of its false positives fall on those disputed cases, a concentration that holds across all ten models we evaluate. Expert disagreement thus signals uncertainty in the task and in its gold label, which aggregate metrics hide. This should not block useful tools, however: reframed as top-k ranking with multi-model consensus, the same signals reach 76% precision for the top-200 candidates of the benchmark without supervision.