# Extracting Norms from Contracts Via ChatGPT: Opportunities and Challenges

arXiv: https://arxiv.org/abs/2404.02269 · HTML: https://mlsanity.com/paper/2404.02269/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 54 references that are in the map.
- First posted on arXiv 2 Apr 2024.

## Builds on (most cited first, 21 of 21)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference](https://mlsanity.com/paper/1704.05426/) (2017)
- [Sparks of Artificial General Intelligence: Early experiments with GPT-4](https://mlsanity.com/paper/2303.12712/) (2023)
- [Know What You Don't Know: Unanswerable Questions for SQuAD](https://mlsanity.com/paper/1806.03822/) (2018)
- [CUAD: An Expert-Annotated NLP Dataset for Legal Contract Review](https://mlsanity.com/paper/2103.06268/) (2021)
- [ContractNLI: A Dataset for Document-level Natural Language Inference for Contracts](https://mlsanity.com/paper/2110.01799/) (2021)
- [ChatGPT and Software Testing Education: Promises & Perils](https://mlsanity.com/paper/2302.03287/) (2023)
- [A Benchmark for Lease Contract Review](https://mlsanity.com/paper/2010.10386/) (2020)
- [ChatGPT for Programming Numerical Methods](https://mlsanity.com/paper/2303.12093/) (2023)
- [Processing Long Legal Documents with Pre-trained Transformers: Modding LegalBERT and Longformer](https://mlsanity.com/paper/2211.00974/) (2022)
- [Obligation and Prohibition Extraction Using Hierarchical RNNs](https://mlsanity.com/paper/1805.03871/) (2018)
- [An Automated Framework for the Extraction of Semantic Legal Metadata from Legal Texts](https://mlsanity.com/paper/2001.11245/) (2020)
- [Neural Contract Element Extraction Revisited: Letters from Sesame Street](https://mlsanity.com/paper/2101.04355/) (2021)
- [Governance of Autonomous Agents on the Web: Challenges and Opportunities](https://mlsanity.com/paper/2202.02574/) (2022)
- [ConReader: Exploring Implicit Relations in Contracts for Contract Clause Extraction](https://mlsanity.com/paper/2210.08697/) (2022)
- [Hercule: Representing and Reasoning about Norms as a Foundation for Declarative Contracts over Blockchain](https://mlsanity.com/paper/2104.08355/) (2021)

## Built on by (most cited first, 2 of 2)

- [Evolution of Cooperation in LLM-Agent Societies: A Preliminary Study Using Different Punishment Strategies](https://mlsanity.com/paper/2504.19487/) (2025)
- [Social Norm Reasoning in Multimodal Language Models: An Evaluation](https://mlsanity.com/paper/2603.03590/) (2026)

## Abstract (arXiv)

We investigate the effectiveness of ChatGPT in extracting norms from contracts. Norms provide a natural way to engineer multiagent systems by capturing how to govern the interactions between two or more autonomous parties. We extract norms of commitment, prohibition, authorization, and power, along with associated norm elements (the parties involved, antecedents, and consequents) from contracts. Our investigation reveals ChatGPT's effectiveness and limitations in norm extraction from contracts. ChatGPT demonstrates promising performance in norm extraction without requiring training or fine-tuning, thus obviating the need for annotated data, which is not generally available in this domain. However, we found some limitations of ChatGPT in extracting these norms that lead to incorrect norm extractions. The limitations include oversight of crucial details, hallucination, incorrect parsing of conjunctions, and empty norm elements. Enhanced norm extraction from contracts can foster the development of more transparent and trustworthy formal agent interaction specifications, thereby contributing to the improvement of multiagent systems.