# Cross-Lingual Knowledge Transfer for Clinical Phenotyping

arXiv: https://arxiv.org/abs/2208.01912 · HTML: https://mlsanity.com/paper/2208.01912/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 36 references that are in the map.
- First posted on arXiv 3 Aug 2022.

## Builds on (most cited first, 19 of 19)

- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [Cross-lingual Language Model Pretraining](https://mlsanity.com/paper/1901.07291/) (2019)
- [Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing](https://mlsanity.com/paper/2007.15779/) (2020)
- [XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalization](https://mlsanity.com/paper/2003.11080/) (2020)
- [MAD-X: An Adapter-Based Framework for Multi-Task Cross-Lingual Transfer](https://mlsanity.com/paper/2005.00052/) (2020)
- [AdapterHub: A Framework for Adapting Transformers](https://mlsanity.com/paper/2007.07779/) (2020)
- [Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyond](https://mlsanity.com/paper/1812.10464/) (2018)
- [XCOPA: A Multilingual Dataset for Causal Commonsense Reasoning](https://mlsanity.com/paper/2005.00333/) (2020)
- [BEHRT: Transformer for Electronic Health Records](https://mlsanity.com/paper/1907.09538/) (2019)
- [Translation Artifacts in Cross-lingual Transfer Learning](https://mlsanity.com/paper/2004.04721/) (2020)
- [Lightweight Adapter Tuning for Multilingual Speech Translation](https://mlsanity.com/paper/2106.01463/) (2021)
- [Improving Multilingual Models with Language-Clustered Vocabularies](https://mlsanity.com/paper/2010.12777/) (2020)
- [Clinical Outcome Prediction from Admission Notes using Self-Supervised Knowledge Integration](https://mlsanity.com/paper/2102.04110/) (2021)
- [GREEK-BERT: The Greeks visiting Sesame Street](https://mlsanity.com/paper/2008.12014/) (2020)
- [Clinical Text Classification with Rule-based Features and Knowledge-guided Convolutional Neural Networks](https://mlsanity.com/paper/1807.07425/) (2018)
- [Phenotyping of Clinical Notes with Improved Document Classification Models Using Contextualized Neural Language Models](https://mlsanity.com/paper/1910.13664/) (2019)
- [Should we Stop Training More Monolingual Models, and Simply Use Machine Translation Instead?](https://mlsanity.com/paper/2104.10441/) (2021)
- [Biomedical and Clinical Language Models for Spanish: On the Benefits of Domain-Specific Pretraining in a Mid-Resource Scenario](https://mlsanity.com/paper/2109.03570/) (2021)
- [Self-Supervised Detection of Contextual Synonyms in a Multi-Class Setting: Phenotype Annotation Use Case](https://mlsanity.com/paper/2109.01935/) (2021)

## Built on by (most cited first, 2 of 2)

- [Evaluating GPT-4 and ChatGPT on Japanese Medical Licensing Examinations](https://mlsanity.com/paper/2303.18027/) (2023)
- [A Systematic Survey of Natural Language Processing for the Greek Language](https://mlsanity.com/paper/2407.09861/) (2024)

## Abstract (arXiv)

Clinical phenotyping enables the automatic extraction of clinical conditions from patient records, which can be beneficial to doctors and clinics worldwide. However, current state-of-the-art models are mostly applicable to clinical notes written in English. We therefore investigate cross-lingual knowledge transfer strategies to execute this task for clinics that do not use the English language and have a small amount of in-domain data available. We evaluate these strategies for a Greek and a Spanish clinic leveraging clinical notes from different clinical domains such as cardiology, oncology and the ICU. Our results reveal two strategies that outperform the state-of-the-art: Translation-based methods in combination with domain-specific encoders and cross-lingual encoders plus adapters. We find that these strategies perform especially well for classifying rare phenotypes and we advise on which method to prefer in which situation. Our results show that using multilingual data overall improves clinical phenotyping models and can compensate for data sparseness.