# Yes, BM25 is a Strong Baseline for Legal Case Retrieval

arXiv: https://arxiv.org/abs/2105.05686 · HTML: https://mlsanity.com/paper/2105.05686/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 20 references that are in the map.
- First posted on arXiv 26 Apr 2021.

## Builds on (most cited first, 6 of 6)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [MS MARCO: A Human Generated MAchine Reading COmprehension Dataset](https://mlsanity.com/paper/1611.09268/) (2016)
- [Pyserini: An Easy-to-Use Python Toolkit to Support Replicable IR Research with Sparse and Dense Representations](https://mlsanity.com/paper/2102.10073/) (2021)
- [Rapidly Deploying a Neural Search Engine for the COVID-19 Open Research Dataset: Preliminary Thoughts and Lessons Learned](https://mlsanity.com/paper/2004.05125/) (2020)
- [IITP@COLIEE 2019: Legal Information Retrieval using BM25 and BERT](https://mlsanity.com/paper/2104.08653/) (2021)
- [Cross-domain Retrieval in the Legal and Patent Domains: a Reproducibility Study](https://mlsanity.com/paper/2012.11405/) (2020)

## Built on by (most cited first, 24 of 24)

- [A Reasoning-Focused Legal Retrieval Benchmark](https://mlsanity.com/paper/2505.03970/) (2025)
- [The Chronicles of RAG: The Retriever, the Chunk and the Generator](https://mlsanity.com/paper/2401.07883/) (2024)
- [JuDGE: Benchmarking Judgment Document Generation for Chinese Legal System](https://mlsanity.com/paper/2503.14258/) (2025)
- [U-CREAT: Unsupervised Case Retrieval using Events extrAcTion](https://mlsanity.com/paper/2307.05260/) (2023)
- [nigam@COLIEE-22: Legal Case Retrieval and Entailment using Cascading of Lexical and Semantic-based models](https://mlsanity.com/paper/2204.07853/) (2022)
- [DAPR: A Benchmark on Document-Aware Passage Retrieval](https://mlsanity.com/paper/2305.13915/) (2023)
- [Billions of Parameters Are Worth More Than In-domain Training Data: A case study in the Legal Case Entailment Task](https://mlsanity.com/paper/2205.15172/) (2022)
- [Generator-Retriever-Generator Approach for Open-Domain Question Answering](https://mlsanity.com/paper/2307.11278/) (2023)
- [Black-Box Analysis: GPTs Across Time in Legal Textual Entailment Task](https://mlsanity.com/paper/2309.05501/) (2023)
- [CoCoLex: Confidence-guided Copy-based Decoding for Grounded Legal Text Generation](https://mlsanity.com/paper/2508.05534/) (2025)
- [LegalSearchLM: Rethinking Legal Case Retrieval as Legal Elements Generation](https://mlsanity.com/paper/2505.23832/) (2025)
- [Evaluating Legal Reasoning Traces with Legal Issue Tree Rubrics](https://mlsanity.com/paper/2512.01020/) (2025)
- [NOWJ at COLIEE 2023 -- Multi-Task and Ensemble Approaches in Legal Information Processing](https://mlsanity.com/paper/2306.04903/) (2023)
- [LePaRD: A Large-Scale Dataset of Judges Citing Precedents](https://mlsanity.com/paper/2311.09356/) (2023)
- [Enhancing Legal Document Retrieval: A Multi-Phase Approach with Large Language Models](https://mlsanity.com/paper/2403.18093/) (2024)
- [Assessing the Performance Gap Between Lexical and Semantic Models for Information Retrieval With Formulaic Legal Language](https://mlsanity.com/paper/2506.12895/) (2025)
- [ASP2LJ : An Adversarial Self-Play Laywer Augmented Legal Judgment Framework](https://mlsanity.com/paper/2506.18768/) (2025)
- [LLM-based Embedders for Prior Case Retrieval](https://mlsanity.com/paper/2507.18455/) (2025)
- [Comparing Knowledge Injection Methods for LLMs in a Low-Resource Regime](https://mlsanity.com/paper/2508.06178/) (2025)
- [Sequencing Matters: A Generate-Retrieve-Generate Model for Building Conversational Agents](https://mlsanity.com/paper/2311.09513/) (2023)
- [Discourse-Aware Scientific Paper Recommendation via QA-Style Summarization and Multi-Level Contrastive Learning](https://mlsanity.com/paper/2511.03330/) (2025)
- [When Rules Learn: A Self-Evolving Agent for Legal Case Retrieval](https://mlsanity.com/paper/2606.17220/) (2026)
- [Citation Recommendation on Scholarly Legal Articles](https://mlsanity.com/paper/2311.05902/) (2023)
- [GPTs and Language Barrier: A Cross-Lingual Legal QA Examination](https://mlsanity.com/paper/2403.18098/) (2024)

## Abstract (arXiv)

We describe our single submission to task 1 of COLIEE 2021. Our vanilla BM25 got second place, well above the median of submissions. Code is available at https://github.com/neuralmind-ai/coliee.