# LLMs for Legal Subsumption in German Employment Contracts

arXiv: https://arxiv.org/abs/2507.01734 · HTML: https://mlsanity.com/paper/2507.01734/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 18 of its 35 references that are in the map.
- First posted on arXiv 2 Jul 2025.

## Builds on (most cited first, 18 of 18)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference](https://mlsanity.com/paper/2403.04132/) (2024)
- [CUAD: An Expert-Annotated NLP Dataset for Legal Contract Review](https://mlsanity.com/paper/2103.06268/) (2021)
- [CLAUDETTE: an Automated Detector of Potentially Unfair Clauses in Online Terms of Service](https://mlsanity.com/paper/1805.01217/) (2018)
- [Natural Language Processing in the Legal Domain](https://mlsanity.com/paper/2302.12039/) (2023)
- [LawLLM: Law Large Language Model for the US Legal System](https://mlsanity.com/paper/2407.21065/) (2024)
- [A Benchmark for Lease Contract Review](https://mlsanity.com/paper/2010.10386/) (2020)
- [Deconfounding Legal Judgment Prediction for European Court of Human Rights Cases Towards Better Alignment with Experts](https://mlsanity.com/paper/2210.13836/) (2022)
- [Measuring Law Over Time: A Network Analytical Framework with an Application to Statutes and Regulations in the United States and Germany](https://mlsanity.com/paper/2101.11284/) (2021)
- [Complex Societies and the Growth of the Law](https://mlsanity.com/paper/2005.07646/) (2020)
- [Needle Threading: Can LLMs Follow Threads through Near-Million-Scale Haystacks?](https://mlsanity.com/paper/2411.05000/) (2024)
- [Fact-based Court Judgment Prediction](https://mlsanity.com/paper/2311.13350/) (2023)
- [Rethinking Legal Judgement Prediction in a Realistic Scenario in the Era of Large Language Models](https://mlsanity.com/paper/2410.10542/) (2024)
- [The Law and NLP: Bridging Disciplinary Disconnects](https://mlsanity.com/paper/2310.14346/) (2023)
- [Towards Explainability in Legal Outcome Prediction Models](https://mlsanity.com/paper/2403.16852/) (2024)
- [Incorporating Precedents for Legal Judgement Prediction on European Court of Human Rights Cases](https://mlsanity.com/paper/2409.18644/) (2024)
- [Team UTSA-NLP at SemEval 2024 Task 5: Prompt Ensembling for Argument Reasoning in Civil Procedures with GPT4](https://mlsanity.com/paper/2404.01961/) (2024)
- [AI-assisted German Employment Contract Review: A Benchmark Dataset](https://mlsanity.com/paper/2501.17194/) (2025)

## Abstract (arXiv)

Legal work, characterized by its text-heavy and resource-intensive nature, presents unique challenges and opportunities for NLP research. While data-driven approaches have advanced the field, their lack of interpretability and trustworthiness limits their applicability in dynamic legal environments. To address these issues, we collaborated with legal experts to extend an existing dataset and explored the use of Large Language Models (LLMs) and in-context learning to evaluate the legality of clauses in German employment contracts. Our work evaluates the ability of different LLMs to classify clauses as "valid," "unfair," or "void" under three legal context variants: no legal context, full-text sources of laws and court rulings, and distilled versions of these (referred to as examination guidelines). Results show that full-text sources moderately improve performance, while examination guidelines significantly enhance recall for void clauses and weighted F1-Score, reaching 80%. Despite these advancements, LLMs' performance when using full-text sources remains substantially below that of human lawyers. We contribute an extended dataset, including examination guidelines, referenced legal sources, and corresponding annotations, alongside our code and all log files. Our findings highlight the potential of LLMs to assist lawyers in contract legality review while also underscoring the limitations of the methods presented.