# Evaluating the Role of Large Language Models in Legal Practice in India

arXiv: https://arxiv.org/abs/2508.09713 · HTML: https://mlsanity.com/paper/2508.09713/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 35 references that are in the map.
- First posted on arXiv 13 Aug 2025.

## Builds on (most cited first, 9 of 9)

- [LegalBench: A Collaboratively Built Benchmark for Measuring Legal Reasoning in Large Language Models](https://mlsanity.com/paper/2308.11462/) (2023)
- [GPT Takes the Bar Exam](https://mlsanity.com/paper/2212.14402/) (2022)
- [Precedent-Enhanced Legal Judgment Prediction with LLM and Domain-Model Collaboration](https://mlsanity.com/paper/2310.09241/) (2023)
- [LawLLM: Law Large Language Model for the US Legal System](https://mlsanity.com/paper/2407.21065/) (2024)
- [ChatGPT may Pass the Bar Exam soon, but has a Long Way to Go for the LexGLUE benchmark](https://mlsanity.com/paper/2304.12202/) (2023)
- [Better Call GPT, Comparing Large Language Models Against Lawyers](https://mlsanity.com/paper/2401.16212/) (2024)
- [Evaluating AI for Law: Bridging the Gap with Open-Source Solutions](https://mlsanity.com/paper/2404.12349/) (2024)
- [LLM vs. Lawyers: Identifying a Subset of Summary Judgments in a Large UK Case Law Dataset](https://mlsanity.com/paper/2403.04791/) (2024)
- [PILOT: Legal Case Outcome Prediction with Case Law](https://mlsanity.com/paper/2401.15770/) (2024)

## Built on by (most cited first, 2 of 2)

- [AIReg-Bench: Benchmarking Language Models That Assess AI Regulation Compliance](https://mlsanity.com/paper/2510.01474/) (2025)
- [Eskwai for Students: Generative AI Assistant for Legal Education in Ghana](https://mlsanity.com/paper/2605.15380/) (2026)

## Abstract (arXiv)

The integration of Artificial Intelligence(AI) into the legal profession raises significant questions about the capacity of Large Language Models(LLM) to perform key legal tasks. In this paper, I empirically evaluate how well LLMs, such as GPT, Claude, and Llama, perform key legal tasks in the Indian context, including issue spotting, legal drafting, advice, research, and reasoning. Through a survey experiment, I compare outputs from LLMs with those of a junior lawyer, with advanced law students rating the work on helpfulness, accuracy, and comprehensiveness. LLMs excel in drafting and issue spotting, often matching or surpassing human work. However, they struggle with specialised legal research, frequently generating hallucinations, factually incorrect or fabricated outputs. I conclude that while LLMs can augment certain legal tasks, human expertise remains essential for nuanced reasoning and the precise application of law.