# Exploiting LLMs' Reasoning Capability to Infer Implicit Concepts in Legal Information Retrieval

arXiv: https://arxiv.org/abs/2410.12154 · HTML: https://mlsanity.com/paper/2410.12154/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning.
- Builds on 17 of its 38 references that are in the map.
- First posted on arXiv 16 Oct 2024.

## Placed in

- LLM reasoning / [Reasoning for agents, planning and code](https://mlsanity.com/lr/reasoning-for-agents-planning-and-code/)

## Builds on (most cited first, 17 of 17)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [A Survey of Large Language Models](https://mlsanity.com/paper/2303.18223/) (2023)
- [Gemini: A Family of Highly Capable Multimodal Models](https://mlsanity.com/paper/2312.11805/) (2023)
- [Large Language Models in Law: A Survey](https://mlsanity.com/paper/2312.03718/) (2023)
- [Legal Prompting: Teaching a Language Model to Think Like a Lawyer](https://mlsanity.com/paper/2212.01326/) (2022)
- [Query Expansion by Prompting Large Language Models](https://mlsanity.com/paper/2305.03653/) (2023)
- [Legal Prompt Engineering for Multilingual Legal Judgement Prediction](https://mlsanity.com/paper/2212.02199/) (2022)
- [Benchmarking Machine Translation with Cultural Awareness](https://mlsanity.com/paper/2305.14328/) (2023)
- [A Short Survey of Viewing Large Language Models in Legal Aspect](https://mlsanity.com/paper/2303.09136/) (2023)
- [BERT-QE: Contextualized Query Expansion for Document Re-ranking](https://mlsanity.com/paper/2009.07258/) (2020)
- [Can LLMs Reason with Rules? Logic Scaffolding for Stress-Testing and Improving LLMs](https://mlsanity.com/paper/2402.11442/) (2024)
- [Legal Summarisation through LLMs: The PRODIGIT Project](https://mlsanity.com/paper/2308.04416/) (2023)
- [LLaMandement: Large Language Models for Summarization of French Legislative Proposals](https://mlsanity.com/paper/2401.16182/) (2024)
- [CAPTAIN at COLIEE 2023: Efficient Methods for Legal Information Retrieval and Entailment Tasks](https://mlsanity.com/paper/2401.03551/) (2024)
- [NOWJ1@ALQAC 2023: Enhancing Legal Task Performance with Classic Statistical Models and Pre-trained Language Models](https://mlsanity.com/paper/2309.09070/) (2023)

## Built on by (most cited first, 1 of 1)

- [A New Query Expansion Approach via Agent-Mediated Dialogic Inquiry](https://mlsanity.com/paper/2502.08557/) (2025)

## Abstract (arXiv)

Statutory law retrieval is a typical problem in legal language processing, that has various practical applications in law engineering. Modern deep learning-based retrieval methods have achieved significant results for this problem. However, retrieval systems relying on semantic and lexical correlations often exhibit limitations, particularly when handling queries that involve real-life scenarios, or use the vocabulary that is not specific to the legal domain. In this work, we focus on overcoming this weaknesses by utilizing the logical reasoning capabilities of large language models (LLMs) to identify relevant legal terms and facts related to the situation mentioned in the query. The proposed retrieval system integrates additional information from the term--based expansion and query reformulation to improve the retrieval accuracy. The experiments on COLIEE 2022 and COLIEE 2023 datasets show that extra knowledge from LLMs helps to improve the retrieval result of both lexical and semantic ranking models. The final ensemble retrieval system outperformed the highest results among all participating teams in the COLIEE 2022 and 2023 competitions.