# CaseFacts: A Benchmark for Legal Fact-Checking and Precedent Retrieval

arXiv: https://arxiv.org/abs/2601.17230 · HTML: https://mlsanity.com/paper/2601.17230/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 27 references that are in the map.
- First posted on arXiv 23 Jan 2026.

## Builds on (most cited first, 15 of 15)

- [MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers](https://mlsanity.com/paper/2002.10957/) (2020)
- [Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Models](https://mlsanity.com/paper/2506.05176/) (2025)
- [C-Pack: Packed Resources For General Chinese Embeddings](https://mlsanity.com/paper/2309.07597/) (2023)
- [Self-Preference Bias in LLM-as-a-Judge](https://mlsanity.com/paper/2410.21819/) (2024)
- [When Does Pretraining Help? Assessing Self-Supervised Learning for Law and the CaseHOLD Dataset](https://mlsanity.com/paper/2104.08671/) (2021)
- [FEVEROUS: Fact Extraction and VERification Over Unstructured and Structured information](https://mlsanity.com/paper/2106.05707/) (2021)
- [AVeriTeC: A Dataset for Real-world Claim Verification with Evidence from the Web](https://mlsanity.com/paper/2305.13117/) (2023)
- [Synthetic Data Generation Using Large Language Models: Advances in Text and Code](https://mlsanity.com/paper/2503.14023/) (2025)
- [DEFAME: Dynamic Evidence-based FAct-checking with Multimodal Experts](https://mlsanity.com/paper/2412.10510/) (2024)
- [Fine-tuning Large Language Models for Improving Factuality in Legal Question Answering](https://mlsanity.com/paper/2501.06521/) (2025)
- [OpenFactCheck: Building, Benchmarking Customized Fact-Checking Systems and Evaluating the Factuality of Claims and LLMs](https://mlsanity.com/paper/2405.05583/) (2024)
- [Towards Reliable Retrieval in RAG Systems for Large Legal Datasets](https://mlsanity.com/paper/2510.06999/) (2025)
- [Task-Oriented Automatic Fact-Checking with Frame-Semantics](https://mlsanity.com/paper/2501.13288/) (2025)
- [Assessing the Performance Gap Between Lexical and Semantic Models for Information Retrieval With Formulaic Legal Language](https://mlsanity.com/paper/2506.12895/) (2025)
- [Enhancing Health Fact-Checking with LLM-Generated Synthetic Data](https://mlsanity.com/paper/2508.20525/) (2025)

## Built on by (most cited first, 1 of 1)

- [Controlling Authority Retrieval: A Missing Retrieval Objective for Authority-Governed Knowledge](https://mlsanity.com/paper/2604.14488/) (2026)

## Abstract (arXiv)

Automated Fact-Checking has largely focused on verifying general knowledge against static corpora, overlooking high-stakes domains like law where truth is evolving and technically complex. We introduce CaseFacts, a benchmark for verifying colloquial legal claims against U.S. Supreme Court precedents. Unlike existing resources that map formal texts to formal texts, CaseFacts challenges systems to bridge the semantic gap between layperson assertions and technical jurisprudence while accounting for temporal validity. The dataset consists of 6,294 claims categorized as Supported, Refuted, or Overruled. We construct this benchmark using a multi-stage pipeline that leverages Large Language Models (LLMs) to synthesize claims from expert case summaries, employing a novel semantic similarity heuristic to efficiently identify and verify complex legal overrulings. Experiments with state-of-the-art LLMs reveal that the task remains challenging; notably, augmenting models with unrestricted web search degrades performance compared to closed-book baselines due to the retrieval of noisy, non-authoritative precedents. We release CaseFacts to spur research into legal fact verification systems.