# Better Call GPT, Comparing Large Language Models Against Lawyers

arXiv: https://arxiv.org/abs/2401.16212 · HTML: https://mlsanity.com/paper/2401.16212/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- Builds on 2 of its 11 references that are in the map.
- First posted on arXiv 24 Jan 2024.

## Builds on (most cited first, 2 of 2)

- [PaLM 2 Technical Report](https://mlsanity.com/paper/2305.10403/) (2023)
- [LegalBench: A Collaboratively Built Benchmark for Measuring Legal Reasoning in Large Language Models](https://mlsanity.com/paper/2308.11462/) (2023)

## Built on by (most cited first, 26 of 26)

- [Knowledge Conflicts for LLMs: A Survey](https://mlsanity.com/paper/2403.08319/) (2024)
- [SaulLM-7B: A pioneering Large Language Model for Law](https://mlsanity.com/paper/2403.03883/) (2024)
- [A Survey of LLM-Driven AI Agent Communication: Protocols, Security Risks, and Defense Countermeasures](https://mlsanity.com/paper/2506.19676/) (2025)
- [GreekBarBench: A Challenging Benchmark for Free-Text Legal Reasoning and Citations](https://mlsanity.com/paper/2505.17267/) (2025)
- [Evaluating AI for Law: Bridging the Gap with Open-Source Solutions](https://mlsanity.com/paper/2404.12349/) (2024)
- [CEBench: A Benchmarking Toolkit for the Cost-Effectiveness of LLM Pipelines](https://mlsanity.com/paper/2407.12797/) (2024)
- [ACORD: An Expert-Annotated Retrieval Dataset for Legal Contract Drafting](https://mlsanity.com/paper/2501.06582/) (2025)
- [The Hidden Structure -- Improving Legal Document Understanding Through Explicit Text Formatting](https://mlsanity.com/paper/2505.12837/) (2025)
- [AGB-DE: A Corpus for the Automated Legal Assessment of Clauses in German Consumer Contracts](https://mlsanity.com/paper/2406.06809/) (2024)
- [Teaching AI to Handle Exceptions: Supervised Fine-Tuning with Human-Aligned Judgment](https://mlsanity.com/paper/2503.02976/) (2025)
- [Compliance-to-Code: Enhancing Financial Compliance Checking via Code Generation](https://mlsanity.com/paper/2505.19804/) (2025)
- [Exploring Possibilities of AI-Powered Legal Assistance in Bangladesh through Large Language Modeling](https://mlsanity.com/paper/2410.17210/) (2024)
- [When Large Language Models Meet Law: Dual-Lens Taxonomy, Technical Advances, and Ethical Governance](https://mlsanity.com/paper/2507.07748/) (2025)
- [Evaluating the Role of Large Language Models in Legal Practice in India](https://mlsanity.com/paper/2508.09713/) (2025)
- [Mina: A Multilingual LLM-Powered Legal Assistant Agent for Bangladesh for Empowering Access to Justice](https://mlsanity.com/paper/2511.08605/) (2025)
- [Legal-DC: Benchmarking Retrieval-Augmented Generation for Legal Documents](https://mlsanity.com/paper/2603.11772/) (2026)
- [ALKAFI-LLAMA3: Fine-Tuning LLMs for Precise Legal Understanding in Palestine](https://mlsanity.com/paper/2412.14771/) (2024)
- [Lightweight Domain Adaptation of a Large Language Model for Legal Assistance in the Indian Context](https://mlsanity.com/paper/2505.22003/) (2025)
- [Compiled Memory: Not More Information, but More Precise Instructions for Language Agents](https://mlsanity.com/paper/2603.15666/) (2026)
- [A Few Good Clauses: Comparing LLMs vs Domain-Trained Small Language Models on Structured Contract Extraction](https://mlsanity.com/paper/2605.05532/) (2026)
- [Better Bill GPT: Comparing Large Language Models against Legal Invoice Reviewers](https://mlsanity.com/paper/2504.02881/) (2025)
- [Using Large Language Models for Legal Decision-Making in Austrian Value-Added Tax Law: A Comparative Study](https://mlsanity.com/paper/2507.08468/) (2025)
- [Legal Document Summarization: Enhancing Judicial Efficiency through Automation Detection](https://mlsanity.com/paper/2507.18952/) (2025)
- [Legal interpretation and AI: from expert systems to argumentation and LLMs](https://mlsanity.com/paper/2603.05392/) (2026)
- [DECODEM: Data Extraction from Corporate Organizational Documents via Enhanced Methods](https://mlsanity.com/paper/2607.15879/) (2026)
- [InvestigationWorlds: An Agentic Environment for Legal Investigation](https://mlsanity.com/paper/2610.04129/) (2026)

## Abstract (arXiv)

This paper presents a groundbreaking comparison between Large Language Models and traditional legal contract reviewers, Junior Lawyers and Legal Process Outsourcers. We dissect whether LLMs can outperform humans in accuracy, speed, and cost efficiency during contract review. Our empirical analysis benchmarks LLMs against a ground truth set by Senior Lawyers, uncovering that advanced models match or exceed human accuracy in determining legal issues. In speed, LLMs complete reviews in mere seconds, eclipsing the hours required by their human counterparts. Cost wise, LLMs operate at a fraction of the price, offering a staggering 99.97 percent reduction in cost over traditional methods. These results are not just statistics, they signal a seismic shift in legal practice. LLMs stand poised to disrupt the legal industry, enhancing accessibility and efficiency of legal services. Our research asserts that the era of LLM dominance in legal contract review is upon us, challenging the status quo and calling for a reimagined future of legal workflows.