# AGB-DE: A Corpus for the Automated Legal Assessment of Clauses in German Consumer Contracts

arXiv: https://arxiv.org/abs/2406.06809 · HTML: https://mlsanity.com/paper/2406.06809/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 37 references that are in the map.
- First posted on arXiv 10 Jun 2024.

## Builds on (most cited first, 9 of 9)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [LexGLUE: A Benchmark Dataset for Legal Language Understanding in English](https://mlsanity.com/paper/2110.00976/) (2021)
- [CUAD: An Expert-Annotated NLP Dataset for Legal Contract Review](https://mlsanity.com/paper/2103.06268/) (2021)
- [Leak, Cheat, Repeat: Data Contamination and Evaluation Malpractices in Closed-Source LLMs](https://mlsanity.com/paper/2402.03927/) (2024)
- [Neural Legal Judgment Prediction in English](https://mlsanity.com/paper/1906.02059/) (2019)
- [German's Next Language Model](https://mlsanity.com/paper/2010.10906/) (2020)
- [Better Call GPT, Comparing Large Language Models Against Lawyers](https://mlsanity.com/paper/2401.16212/) (2024)
- [Towards an Open Platform for Legal Information](https://mlsanity.com/paper/2005.13342/) (2020)

## Built on by (most cited first, 4 of 4)

- [ContractEval: Benchmarking LLMs for Clause-Level Legal Risk Identification in Commercial Contracts](https://mlsanity.com/paper/2508.03080/) (2025)
- [Evaluating and Benchmarking the System One Model Jev](https://mlsanity.com/paper/2609.37647/) (2026)
- [Acquiescence Bias in Large Language Models](https://mlsanity.com/paper/2509.08480/) (2025)
- [Predicting potentially abusive clauses in Chilean terms of services with natural language processing](https://mlsanity.com/paper/2502.00865/) (2025)

## Abstract (arXiv)

Legal tasks and datasets are often used as benchmarks for the capabilities of language models. However, openly available annotated datasets are rare. In this paper, we introduce AGB-DE, a corpus of 3,764 clauses from German consumer contracts that have been annotated and legally assessed by legal experts. Together with the data, we present a first baseline for the task of detecting potentially void clauses, comparing the performance of an SVM baseline with three fine-tuned open language models and the performance of GPT-3.5. Our results show the challenging nature of the task, with no approach exceeding an F1-score of 0.54. While the fine-tuned models often performed better with regard to precision, GPT-3.5 outperformed the other approaches with regard to recall. An analysis of the errors indicates that one of the main challenges could be the correct interpretation of complex clauses, rather than the decision boundaries of what is permissible and what is not.