# GERNERMED++: Transfer Learning in German Medical NLP

arXiv: https://arxiv.org/abs/2206.14504 · HTML: https://mlsanity.com/paper/2206.14504/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 40 references that are in the map.
- First posted on arXiv 29 Jun 2022.

## Builds on (most cited first, 13 of 13)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [BioBERT: a pre-trained biomedical language representation model for biomedical text mining](https://mlsanity.com/paper/1901.08746/) (2019)
- [fairseq: A Fast, Extensible Toolkit for Sequence Modeling](https://mlsanity.com/paper/1904.01038/) (2019)
- [SciBERT: A Pretrained Language Model for Scientific Text](https://mlsanity.com/paper/1903.10676/) (2019)
- [Publicly Available Clinical BERT Embeddings](https://mlsanity.com/paper/1904.03323/) (2019)
- [Med-BERT: pre-trained contextualized embeddings on large-scale structured electronic health records for disease prediction](https://mlsanity.com/paper/2005.12833/) (2020)
- [Transfer Learning in Biomedical Natural Language Processing: An Evaluation of BERT and ELMo on Ten Benchmarking Datasets](https://mlsanity.com/paper/1906.05474/) (2019)
- [Word Alignment by Fine-tuning Embeddings on Parallel Corpora](https://mlsanity.com/paper/2101.08231/) (2021)
- [SimAlign: High Quality Word Alignments without Parallel Training Data using Static and Contextualized Embeddings](https://mlsanity.com/paper/2004.08728/) (2020)
- [GottBERT: a pure German Language Model](https://mlsanity.com/paper/2012.02110/) (2020)
- [GGPONC: A Corpus of German Medical Text with Rich Metadata Based on Clinical Practice Guidelines](https://mlsanity.com/paper/2007.06400/) (2020)
- [Towards Automated Anamnesis Summarization: BERT-based Models for Symptom Extraction](https://mlsanity.com/paper/2011.01696/) (2020)

## Built on by (most cited first, 3 of 3)

- [Multilingual Clinical NER: Translation or Cross-lingual Transfer?](https://mlsanity.com/paper/2306.04384/) (2023)
- [GeistBERT: Breathing Life into German NLP](https://mlsanity.com/paper/2506.11903/) (2025)
- [Annotated Dataset Creation through General Purpose Language Models for non-English Medical NLP](https://mlsanity.com/paper/2208.14493/) (2022)

## Abstract (arXiv)

We present a statistical model for German medical natural language processing trained for named entity recognition (NER) as an open, publicly available model. The work serves as a refined successor to our first GERNERMED model which is substantially outperformed by our work. We demonstrate the effectiveness of combining multiple techniques in order to achieve strong results in entity recognition performance by the means of transfer-learning on pretrained deep language models (LM), word-alignment and neural machine translation. Due to the sparse situation on open, public medical entity recognition models for German texts, this work offers benefits to the German research community on medical NLP as a baseline model. Since our model is based on public English data, its weights are provided without legal restrictions on usage and distribution. The sample code and the statistical model is available at: https://github.com/frankkramer-lab/GERNERMED-pp