# German BERT Model for Legal Named Entity Recognition

arXiv: https://arxiv.org/abs/2303.05388 · HTML: https://mlsanity.com/paper/2303.05388/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 36 references that are in the map.
- First posted on arXiv 7 Mar 2023.

## Builds on (most cited first, 9 of 9)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Neural Architectures for Named Entity Recognition](https://mlsanity.com/paper/1603.01360/) (2016)
- [Bidirectional LSTM-CRF Models for Sequence Tagging](https://mlsanity.com/paper/1508.01991/) (2015)
- [German's Next Language Model](https://mlsanity.com/paper/2010.10906/) (2020)
- [Portuguese Named Entity Recognition using BERT-CRF](https://mlsanity.com/paper/1909.10649/) (2019)
- [A Dataset of German Legal Documents for Named Entity Recognition](https://mlsanity.com/paper/2003.13016/) (2020)
- [Beheshti-NER: Persian Named Entity Recognition Using BERT](https://mlsanity.com/paper/2003.08875/) (2020)

## Built on by (most cited first, 6 of 6)

- [LexGPT 0.1: pre-trained GPT-J models with Pile of Law](https://mlsanity.com/paper/2306.05431/) (2023)
- [Automatic Anonymization of Swiss Federal Supreme Court Rulings](https://mlsanity.com/paper/2310.04632/) (2023)
- [Challenges and Considerations in Annotating Legal Data: A Comprehensive Overview](https://mlsanity.com/paper/2407.17503/) (2024)
- [Forging GEMs: Advancing Greek NLP through Quality-Based Corpus Curation](https://mlsanity.com/paper/2510.20002/) (2025)
- [Named entity recognition in resumes](https://mlsanity.com/paper/2306.13062/) (2023)
- [Exploring prompts to elicit memorization in masked language model-based named entity recognition](https://mlsanity.com/paper/2405.03004/) (2024)

## Abstract (arXiv)

The use of BERT, one of the most popular language models, has led to improvements in many Natural Language Processing (NLP) tasks. One such task is Named Entity Recognition (NER) i.e. automatic identification of named entities such as location, person, organization, etc. from a given text. It is also an important base step for many NLP tasks such as information extraction and argumentation mining. Even though there is much research done on NER using BERT and other popular language models, the same is not explored in detail when it comes to Legal NLP or Legal Tech. Legal NLP applies various NLP techniques such as sentence similarity or NER specifically on legal data. There are only a handful of models for NER tasks using BERT language models, however, none of these are aimed at legal documents in German. In this paper, we fine-tune a popular BERT language model trained on German data (German BERT) on a Legal Entity Recognition (LER) dataset. To make sure our model is not overfitting, we performed a stratified 10-fold cross-validation. The results we achieve by fine-tuning German BERT on the LER dataset outperform the BiLSTM-CRF+ model used by the authors of the same LER dataset. Finally, we make the model openly available via HuggingFace.