mlsanityData to 7 Oct 2026

Better Call GPT, Comparing Large Language Models Against Lawyers

arXiv 2401.16212 ↗First posted 24 Jan 2024cs.CY cs.CLcited by 25

Not on the map. Why?

Abstract (arXiv)

This paper presents a groundbreaking comparison between Large Language Models and traditional legal contract reviewers, Junior Lawyers and Legal Process Outsourcers. We dissect whether LLMs can outperform humans in accuracy, speed, and cost efficiency during contract review. Our empirical analysis benchmarks LLMs against a ground truth set by Senior Lawyers, uncovering that advanced models match or exceed human accuracy in determining legal issues. In speed, LLMs complete reviews in mere seconds, eclipsing the hours required by their human counterparts. Cost wise, LLMs operate at a fraction of the price, offering a staggering 99.97 percent reduction in cost over traditional methods. These results are not just statistics, they signal a seismic shift in legal practice. LLMs stand poised to disrupt the legal industry, enhancing accessibility and efficiency of legal services. Our research asserts that the era of LLM dominance in legal contract review is upon us, challenging the status quo and calling for a reimagined future of legal workflows.

Key facts. Cited by 25 papers in the map (data through 7 Oct 2026). Builds on 2 of its 11 references that are in the map. First posted on arXiv 24 Jan 2024.

Builds on

2 of 11 references.

References · 2

  1. 2023PaLM 2 Technical Reportcited by 1,154
  2. 2023LegalBench: A Collaboratively Built Benchmark for Measuring Legal Reasoning in Large Language Modelscited by 328

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 7; 2025: 13; 2026: 5.

25 papers.

Sort:

By area:

  1. 2024Knowledge Conflicts for LLMs: A Surveycited by 274
  2. 2024SaulLM-7B: A pioneering Large Language Model for Lawcited by 136
  3. 2025A Survey of LLM-Driven AI Agent Communication: Protocols, Security Risks, and Defense Countermeasurescited by 54
  4. 2025GreekBarBench: A Challenging Benchmark for Free-Text Legal Reasoning and Citationscited by 15
  5. 2024Evaluating AI for Law: Bridging the Gap with Open-Source Solutionscited by 11
  6. 2024CEBench: A Benchmarking Toolkit for the Cost-Effectiveness of LLM Pipelinescited by 11
  7. 2025ACORD: An Expert-Annotated Retrieval Dataset for Legal Contract Draftingcited by 11
  8. 2025The Hidden Structure -- Improving Legal Document Understanding Through Explicit Text Formattingcited by 5
  9. 2024AGB-DE: A Corpus for the Automated Legal Assessment of Clauses in German Consumer Contractscited by 4
  10. 2025Teaching AI to Handle Exceptions: Supervised Fine-Tuning with Human-Aligned Judgmentcited by 4
  11. 2025Compliance-to-Code: Enhancing Financial Compliance Checking via Code Generationcited by 4
  12. 2024Exploring Possibilities of AI-Powered Legal Assistance in Bangladesh through Large Language Modelingcited by 3
13 more
  1. 2025When Large Language Models Meet Law: Dual-Lens Taxonomy, Technical Advances, and Ethical Governancecited by 3
  2. 2025Evaluating the Role of Large Language Models in Legal Practice in Indiacited by 2
  3. 2025Mina: A Multilingual LLM-Powered Legal Assistant Agent for Bangladesh for Empowering Access to Justicecited by 2
  4. 2026Legal-DC: Benchmarking Retrieval-Augmented Generation for Legal Documentscited by 2
  5. 2024ALKAFI-LLAMA3: Fine-Tuning LLMs for Precise Legal Understanding in Palestinecited by 1
  6. 2025Lightweight Domain Adaptation of a Large Language Model for Legal Assistance in the Indian Contextcited by 1
  7. 2026A Few Good Clauses: Comparing LLMs vs Domain-Trained Small Language Models on Structured Contract Extractioncited by 1
  8. 2025Better Bill GPT: Comparing Large Language Models against Legal Invoice Reviewerscited by 0
  9. 2025Using Large Language Models for Legal Decision-Making in Austrian Value-Added Tax Law: A Comparative Studycited by 0
  10. 2025Legal Document Summarization: Enhancing Judicial Efficiency through Automation Detectioncited by 0
  11. 2026Legal interpretation and AI: from expert systems to argumentation and LLMscited by 0
  12. 2026Compiled Memory: Not More Information, but More Precise Instructions for Language Agentscited by 0
  13. 2026DECODEM: Data Extraction from Corporate Organizational Documents via Enhanced Methodscited by 0