# ChatGPT Unveils Its Limits: Principles of Law Deliver Checkmate

arXiv: https://arxiv.org/abs/2510.19261 · HTML: https://mlsanity.com/paper/2510.19261/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 2 of its 17 references that are in the map.
- First posted on arXiv 22 Oct 2025.

## Builds on (most cited first, 2 of 2)

- [Humans decompose tasks by trading off utility and computational cost](https://mlsanity.com/paper/2211.03890/) (2022)
- [Semi-automated checking for regulatory compliance in e-Health](https://mlsanity.com/paper/2110.07710/) (2021)

## Abstract (arXiv)

This study examines the performance of ChatGPT with an experiment in the legal domain. We compare the outcome with it a baseline using regular expressions (Regex), rather than focusing solely on the assessment against human performance. The study reveals that even if ChatGPT has access to the necessary knowledge and competencies, it is unable to assemble them, reason through, in a way that leads to an exhaustive result. This unveils a major limitation of ChatGPT. Intelligence encompasses the ability to break down complex issues and address them according to multiple required competencies, providing a unified and comprehensive solution. In the legal domain, one of the most crucial tasks is reading legal decisions and extracting key passages condensed from principles of law (PoLs), which are then incorporated into subsequent rulings by judges or defense documents by lawyers. In performing this task, artificial intelligence lacks an all-encompassing understanding and reasoning, which makes it inherently limited. Genuine intelligence, remains a uniquely human trait, at least in this particular field.