# Enhancing Legal Document Retrieval: A Multi-Phase Approach with Large Language Models

arXiv: https://arxiv.org/abs/2403.18093 · HTML: https://mlsanity.com/paper/2403.18093/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 11 references that are in the map.
- First posted on arXiv 26 Mar 2024.

## Builds on (most cited first, 7 of 7)

- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Is ChatGPT Good at Search? Investigating Large Language Models as Re-Ranking Agents](https://mlsanity.com/paper/2304.09542/) (2023)
- [Yes, BM25 is a Strong Baseline for Legal Case Retrieval](https://mlsanity.com/paper/2105.05686/) (2021)
- [Attentive Deep Neural Networks for Legal Document Retrieval](https://mlsanity.com/paper/2212.13899/) (2022)
- [NOWJ at COLIEE 2023 -- Multi-Task and Ensemble Approaches in Legal Information Processing](https://mlsanity.com/paper/2306.04903/) (2023)
- [CAPTAIN at COLIEE 2023: Efficient Methods for Legal Information Retrieval and Entailment Tasks](https://mlsanity.com/paper/2401.03551/) (2024)
- [NeCo@ALQAC 2023: Legal Domain Knowledge Acquisition for Low-Resource Languages through Data Enrichment](https://mlsanity.com/paper/2309.05500/) (2023)

## Built on by (most cited first, 3 of 3)

- [Hybrid LLM/Rule-based Approaches to Business Insights Generation from Structured Data](https://mlsanity.com/paper/2404.15604/) (2024)
- [Optimizing Legal Document Retrieval in Vietnamese with Semi-Hard Negative Mining](https://mlsanity.com/paper/2507.14619/) (2025)
- [Legal Document Summarization: Enhancing Judicial Efficiency through Automation Detection](https://mlsanity.com/paper/2507.18952/) (2025)

## Abstract (arXiv)

Large language models with billions of parameters, such as GPT-3.5, GPT-4, and LLaMA, are increasingly prevalent. Numerous studies have explored effective prompting techniques to harness the power of these LLMs for various research problems. Retrieval, specifically in the legal data domain, poses a challenging task for the direct application of Prompting techniques due to the large number and substantial length of legal articles. This research focuses on maximizing the potential of prompting by placing it as the final phase of the retrieval system, preceded by the support of two phases: BM25 Pre-ranking and BERT-based Re-ranking. Experiments on the COLIEE 2023 dataset demonstrate that integrating prompting techniques on LLMs into the retrieval system significantly improves retrieval accuracy. However, error analysis reveals several existing issues in the retrieval system that still need resolution.