# Enhancing Logical Reasoning in Large Language Models to Facilitate Legal Applications

arXiv: https://arxiv.org/abs/2311.13095 · HTML: https://mlsanity.com/paper/2311.13095/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning.
- Builds on 11 of its 18 references that are in the map.
- First posted on arXiv 22 Nov 2023.

## Placed in

- LLM reasoning / [Reasoning for agents, planning and code](https://mlsanity.com/lr/reasoning-for-agents-planning-and-code/)

## Builds on (most cited first, 11 of 11)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Towards Reasoning in Large Language Models: A Survey](https://mlsanity.com/paper/2212.10403/) (2022)
- [Selection-Inference: Exploiting Large Language Models for Interpretable Logical Reasoning](https://mlsanity.com/paper/2205.09712/) (2022)
- [Large Language Models as Tax Attorneys: A Case Study in Legal Capabilities Emergence](https://mlsanity.com/paper/2306.07075/) (2023)
- [Boosting Logical Reasoning in Large Language Models through a New Framework: The Graph of Thought](https://mlsanity.com/paper/2308.08614/) (2023)
- [Catala: A Programming Language for the Law](https://mlsanity.com/paper/2103.03198/) (2021)
- [Improved Logical Reasoning of Language Models via Differentiable Symbolic Programming](https://mlsanity.com/paper/2305.03742/) (2023)
- [How well do SOTA legal reasoning models support abductive reasoning?](https://mlsanity.com/paper/2304.06912/) (2023)
- [A negation detection assessment of GPTs: analysis with the xNot360 dataset](https://mlsanity.com/paper/2306.16638/) (2023)
- [A Summary of the ALQAC 2021 Competition](https://mlsanity.com/paper/2204.10717/) (2022)

## Built on by (most cited first, 6 of 6)

- [(A)I Am Not a Lawyer, But...: Engaging Legal Experts towards Responsible LLM Policies for Legal Advice](https://mlsanity.com/paper/2402.01864/) (2024)
- [DiLA: Enhancing LLM Tool Learning with Differential Logic Layer](https://mlsanity.com/paper/2402.11903/) (2024)
- [Crossing Linguistic Horizons: Finetuning and Comprehensive Evaluation of Vietnamese Large Language Models](https://mlsanity.com/paper/2403.02715/) (2024)
- [Town Hall Debate Prompting: Enhancing Logical Reasoning in LLMs through Multi-Persona Interaction](https://mlsanity.com/paper/2502.15725/) (2025)
- [Balancing Exploration and Exploitation in LLM using Soft RLLF for Enhanced Negation Understanding](https://mlsanity.com/paper/2403.01185/) (2024)
- [Delta1 with LLM: symbolic and neural integration for credible and explainable reasoning](https://mlsanity.com/paper/2603.12953/) (2026)

## Abstract (arXiv)

Language serves as a vehicle for conveying thought, enabling communication among individuals. The ability to distinguish between diverse concepts, identify fairness and injustice, and comprehend a range of legal notions fundamentally relies on logical reasoning. Large Language Models (LLMs) attempt to emulate human language understanding and generation, but their competency in logical reasoning remains limited. This paper seeks to address the philosophical question: How can we effectively teach logical reasoning to LLMs while maintaining a deep understanding of the intricate relationship between language and logic? By focusing on bolstering LLMs' capabilities in logical reasoning, we aim to expand their applicability in law and other logic-intensive disciplines. To this end, we propose a Reinforcement Learning from Logical Feedback (RLLF) approach, which serves as a potential framework for refining LLMs' reasoning capacities. Through RLLF and a revised evaluation methodology, we explore new avenues for research in this domain and contribute to the development of LLMs capable of handling complex legal reasoning tasks while acknowledging the fundamental connection between language and logic.